Create a log of all urls requested by Firefox with HTTP Request Logger
HTTP Request Logger is a lightweight add-on for the Firefox web browser that logs all urls connections are made to while the browser is running.
Most of what happens when sites get loaded in a web browser is handled in the background. You type the address of the site you want to visit or click on it, and the browser does the rest.
It may load data from a dozen or so resources in the background to display the contents of that one page.
If you run security software you may be aware of some of those connections, and the same is true if you open the browser's Developer Tools and display the network tab there while you are loading sites.
But those may not display all connections made and even if they do, they usually only echo what is happening in the active tab.
HTTP Request Logger was designed by its author to "track and discover a malicious web page" that exploited a Java vulnerability.
Nothing changes after you install the add-on in Firefox. There is no icon and if you check the addons manager you will notice that there are not any options either.
The add-on page on the Mozilla website fails to mention where the log is saved to (which happens automatically right after installation). I found it on the desktop after some searching, it is called http-request-log.txt.
The add-on writes all urls to the log that get accessed by Firefox while it is up and running. That's interesting for auditing purposes but also development or security.
For instance, you could browse several pages of a website in Firefox to make sure all of them are whitelisted in a proxy or security software so that they are not blocked from access.
Security researchers may use the add-on to detect malicious extensions installed in Firefox or attacks coming from certain websites.
The log is basic as it lists urls, the referrer, and method (get or post). There is no timestamp for instance which could help locate certain requests in the browser quickly.
Since all requests are saved to a text file, basic text file handling including searching through its contents are available.
It is unfortunate that HTTP Request Logger has not been updated since 2011. While that is the case, it works just fine for the time being.
Closing Words
HTTP Request Logger is a useful add-on for Firefox that provides valuable information. While designed for security research, it may be useful to add-on and web developers as well as administrators as well.
problem: the log file generated when palemoon starts, is on the desktop in linuxmint. I ain’t really a big coder or nothing, but I got Midnight Commander, and willing to poke around.
solution: I edited the httpRequestLogger.js (hint it’s inside of the file called http-request-logger@prekageo.xpi)
see where it says “Desk” ….
from..
var file = Components.classes[“@mozilla.org/file/directory_service;1”].getService(Components.interfaces.nsIProperties).get(“Desk”, Components.interfaces.nsIFile);
to.. “Home”
var file = Components.classes[“@mozilla.org/file/directory_service;1”].getService(Components.interfaces.nsIProperties).get(“Home”, Components.interfaces.nsIFile);
– with no coding skills, I GUESSED that “Home” would work. ;o) sure enough
this puts the logfile in your /home/name or ~ dir ..
NOTE if you edit and SAVE THIS FILE WITH IT CHANGED FROM Desk to Home the log file WONT BE ON YOUR DESKTOP it will be in your HOME DIRECTORY.
now I point MY
Eterm -O –viewport-mode –geometry 299×10 –no-cursor –buttonbar no –scrollbar no -f red -x &
tail -f ~/http-request-log.txt
~The Aimless Wanderer
Firefox should natively log all http requests similar to apache server logging. At least it should have option to select how much logging user wants / prefers / requires. Nowadays none of the browsers out of the box come even close to being anything like a tool, but more like a trojan horse.
Martin, take care to not instill a FALSE sense of omniscience. Neither of these These statements are cut-and-dry true: “logs all urls connections are made to” and “writes all urls to the log that get accessed by Firefox”
The HTTP Request Logger extension is blind to: FTP requests, requests occurring within the context of a websocket stream(e.g. webRTC), requests occurring within the context of an SPDY connection, as well as requests employing any other non-http/s protocol (er, scheme).
As a long-time user of this extension, I have witnessed it logging the following HTTP/1.1 methods: GET, PUT, DELETE, POST. Never have I witnessed it logging instances of requests employing OPTIONS, TRACE, nor CONNECT methods (perhaps the extension would log these, but they occur so rarely that I’ve never noticed them in my logfiles).
try https://addons.mozilla.org/en-US/firefox/addon/httpfox/
I rated HTTP Request Logger on April.17, glad to see it here now. :) Simple way to log Firefox requests.
I know it’s off-topic, but I just found nice addon I want to share with firefoxers: https://addons.mozilla.org/firefox/addon/squared-australis-tabs/
I use Classic Theme Restorer for that but alternatives are always handy.
I use Nirsoft‘s excellent TcpLogView when I wish to have a real-time list of all TCP requests, and/but it is system wide.
I read here “If you run security software you may be aware of some of those connections[…]But those may not display all connections made and even if they do, they usually only echo what is happening in the active tab.”
Echoing only the active tab is understandable, but what puzzles me here is that all connections may not be displayed. This is information for me.
Tom, I meant Developer Tools specifically with that as they only allow you to monitor the network connections of the active tab.
OK, Martin. As I had understood it, restricted to the active tab (or not), some connections could not be displayed depending on the security software. Because if this was the case the impact would be considerable, like “black” connections leaving the security software/application/extension blind. As I said this would have been for me information, but no surprise… who ever knows what is known?