Rip Websites with HTTrack Website Copier
HTTRack is a free porgram for Windows and various Unix based operating systems that you can use to copy contents of a website to your local system. It enables you to download all pages and information of a website to the local system. This is commonly referred to as ripping websites. While that's its greatest strength, you can use it as well to quickly download only a single page of a website, or a category instead.
It is a program for experienced users and those who do not mind spending time exploring all the options that it makes available. It offers many configuration options which may look intimidating at first, but if you get through that initial phase, you will certainly appreciate what it has to offer. Most of the settings are optional and help you deal with special case websites that use lots of scripting, dynamically generated pages or require authentication.
A question that may come up is why you would want to rip a website to your local system? There are many reasons for this. Maybe you want to make the contents of the site available for offline browsing. This can be useful if a PC you need the information on has no Internet connection, or at least no permanent connection. It can also be useful if you know or fear that a site may be taken offline in the near future. The program can help you preserve the information by downloading them all to your system.You can last but not least use it to create a local backup of your own site, even though there are usually other options available in this regard.Saving websites to the local computer
- Once you have installed the application run it and click next on the first screen.
- Name your project and assign a category to it (optional). I recommend you use the website's name here.
- The base path is the location where the website will be stored in. Make sure you have enough webspace available on the drive. Click next afterwards.
- You can now enter one or multiple web addresses in a form that you want to process. You can alternatively load a text file that contains a list of urls into the program.
- The action defines what you want the program to do with the urls. The default action is to download websites, but you can change it to update an existing download, test links on the site and a variety of others. Usually, download web site(s) is the right choice here.
- Click on set options to define preferences. This is important and should not be skipped.
- Important preference tabs are limits, which you use to define the maximum mirroring depth (based on links that the program will follow), and scan rules which you can use to include or exclude select links or data types.
- I recommend that you go through the other tabs here as well to get a basic understanding of the program's functionality. Most can be kept at their default levels though.
- You can adjust connection parameters on the next page. Here you can for instance select to shutdown the PC when finished, or disconnect the Internet connection.
The HTTrack website offers a step by step guide that you can use to get to know the program and the core of its features. This should suffice to rip your first website. HTTrack is available for Windows and Unix, Linux & BSD.
The best way to get started with HTTrack is to check out the manual posted on the site which walks you through copying your first website with the help of the program. You can also check out our tutorials on the subject, e.g. how to save websites to your hard drive or how to rip most websites.
Tips
- The program generates a log file whenever it runs an operation. Use it to find errors and issues and adjust the project accordingly.
- You can download 32-bit or 64-bit versions for Windows from the developer website. The program is also being made available as a portable version. Make sure you run WinHTTrack.
- The program supports the https (SSL) protocol.
- The "get files near links" option enables you to download files hosted on third party websites without configuring the program to crawl those third party sites as well.
@Martin Brinkmann: with all due respect, this is now such a basic function, esp. for people reading your articles, that this article is like explaining to the readers how to calculate 1+1.
Yes, I have to agree with this. I can’t imagine that there are many spreadsheet users who are computer literate enough to even casually read ghacks, but don’t know how to use a SUM function.
@Klass Vaak & Herman Cost
Ever thought that people might google it so this artice might show up in the search results. So..no it’s not useless. I think it’s a good strategy explaining stuff even if it’s just basic things. Every day there is a new user on the internet searching for this kind of tutorial.
@Malte: you make a good point, in principle. Nevertheless, nowadays telling people how to do a sum calculation, the simplest of the simple, in a spreadsheet is like teaching grandma to suck eggs.
If say an elderly person would be sitting in front of a spreadsheet wondering how to sum a number of cells, that person is unlikely to look it up on internet, but would ask one of her/his children.
And in any case, Ghacks is not a spreadsheet tricks and tips site, so it would not show up high in the ranking in a Google search.
I`m an elderly person. 71.
My experience in computers spans 50 years, focused on Unix internals, advanced degrees.
I look to the internet for answers, since my children hate me and wouldnt tell me “jack”, even if they knew the answer.
I dont know how to “suck eggs in principle”, or anything else for that matter. I’m retired and not interested in learning. Some other guy learned “the theory of sucking things.”
Never needed spreadsheets, always too darn busy. Thanks for this info. Now I know how to sum two cells.
Everyone starts somewhere.
I am an elderly person and I can use sum, sumif, sumifs and so on. My kids come to me for knowledge on how to use excel and other programs ;)
But then, I was in IT my entire career before I retired :)
There are basic sites, and there are advanced sites. GHacks used to be more for people with technical knowledge. You can’t be everything to everyone though that seems to be the trend these days. Since it’s all about clicks and click bait the wider the topics the more you can remain “relevant”…
However, that mentality tends to alienate the more technical people. I find myself spending less time here every month. It won’t be long before I stop coming here altogether.
I have no mobile or wireless technology.
Anything google or apple is insidious to me. And that accounts now for most of the content here.
As to this article. Avast, and all of their products are untrustworthy. There has been much controversy and negativity regarding their products. I would think a reputable site would be more careful in what they publish… err, sorry, advertise.
@Klaas Vaak: In Martin’s defense, it’s the day after Christmas, and some eggnog recipes pack a *real wallop*. Maybe we’re lucky he could type at all. ;-) More seriously, I don’t use Calc super-often, I typically type in the =SUM() function manually, and I’d actually *forgotten* where the Σ button is located in Calc. (The last time I was a heavy spreadsheet user, it was in an older version of Excel and I’m pretty sure the Σ button was somewhere on one of the “regular” toolbars.) So, the article actually did have a small payoff for me. Plus, there might be other intermediate or advanced users of other types of programs (graphics, audio, video, word-processing) who have never used a spreadsheet in their lives. It’s kind of hard to imagine, I know, but it’s possible.
You were not born with the knowledge you have now. You once had to learn too.
So get down off your high horse, grow up and show a little respect for others who need details.
Overall LO has become a good SW, what prevents me to use it is:
https://wiki.documentfoundation.org/Development/Calc/XMLSource
“it will not store the information about the data source once the data is imported”
Until that functionality is implemented I’m forced to use M$ Office (2003).
@SpywareFan: an interesting alternative for M$ Office is Softmaker’s FreeOffice.
@Klaas Vaak
I have the Softmaker’s Free Office Suite in my computer – – and I LOVE IT!
intelligencia
In the FreeOffice manuals there is no reference to XML data mapping functions.
Another vote for Softmaker’s Free Office, a very nice alternative to MS Office.
Yes, I use functions in Calc: Average, Min and Max. I use the mouse to select the cells.
I don’t usually have to use Sum.
Thanks for the info, and could we have more tips for LibreOffice?
I find that the documentation does not keep up with the changes in LibreOffice.
Also, it is hard to find info in the help documentation.
Thanks Martin for all your great work you put in to make our live easier. Have a nice and healthy 2021.
To all those complaining about this being a simple ‘101’ function: it’s Martin’s blog, he can write a deep-dive review of Windows Calculator if he so wishes. Do you pay his salary or something?
@ShintoPlasm: yes, it is Martin’s blog, but it is a public blog with a comments section, which means he invites people to comment. And having been an avid reader for several years now, I am pretty sure Martin does not expect commenters to agree with him all the time.
You do not help him with agreeing with everything willy-nilly and “honouring” him with praise. Martin, like everyone else in the world, is not perfect, therefore he can only learn from constructive criticism.
Unfortunately you look at constructive criticism as a complaint. I do not agree with you on that and will keep making constructive criticism because I believe in keeping this website one of the best ones – refraining from constructive criticism won’t do that.
Absolutely. People could rather than leaving constructive criticism just stop visiting the site. How is that in the best interest of the site’s authors?
A new way to sum with the latest version is to place the cursor under the numbers to be summed, then press Alt-=, then ENTER.
The SUM function doesn’t work. It just keeps putting the same total no matter what figures I enter.
I would like to know how to make the sum function work. I know how it should work and it works in excel. But when I put the sum function in, it will not make changes when I edit and change the data, the total remains the same
Ditto, Marty. Everyone starts somewhere. Thanks.
Instead of being snarky, maybe you could be helpful. I’d like to add two numbers in a cell;
In Excel, it would be
=5+6
I try that in this piece of crap software, and get a message “Invalid value.”
I used Excel a lot when I was working. Retired now seven years. Excel was great, especially for macros. I now use LibreOffice, which works similarly but the macros are difficult in comparrison to Excel. I couldn’t find anything on macros for Free Office, which was disappointing, so I uninstalled it. I find Zoho to be the most similar to Excel but there you’re stuck with your spreadsheets in the cloud which I didn’t like.
Um… When was this article posted? The date says today (August 23, 2023), but I’m seeing comments from 3 years ago.
I was going to amuse myself and check out the comments for this Avast AV sponsored post since there were so many comments. I thought it funny since they have an article bashing the product in 2019.
What do I see when I go to the comments? Ghacks pulled an “Amazon” and replaced an article on Excel SUM functions with a sponsored post to make it look like a good article.
Anyone and everyone who even has a clue about tech knows that Avast is utter garbage and focuses on spying on you and spamming you with ads these days. Just check out the bad article from 2019 on this very site! If you run Windows 10/11, you already have better antivirus than Avast built-in to Windows.
@graham
surely more importantly is why an ad for dodgy anti-virus has the whole thread on office suites instead.. something in the db is messed up.
On a slight tangent, does foobar2000 have a built in lyrics plug in? I mostly used Linux and Deadbeef, the closest alternative to Foobar there has a very old(and kinda broke) plug in.
If you want news then add this line in uBlock Origin:
ghacks.net##.hentry,.home-posts,.home-category-post:not(:has-text(/Martin Brinkmann|Mike Turcotte|Ashwin/))
@News filter,
> ghacks.net##.hentry,.home-posts,.home-category-post:not(:has-text(/Martin Brinkmann|Mike Turcotte|Ashwin/))
Thanks for the useful information.
Added line to My filters in uBlock Origin.
Magnificently,
I was able to clean up “all articles” by other authors on the Ghacks site.
It’s refreshing to be able to see only useful articles (instead of being buried among inferior articles) by clearing out the obtrusive articles.
Important note: changing policies through the program doesn’t save them automatically. You need to hit Ctrl+S (or go to File > Sve Policies)
What silly click bait article. Even the actual article is ridiculous.
“The launch of HarmonyOS for PC could pose a serious threat to Windows.”
Who cares what people in China use.
Don’t do drugs before working maybe.
What’s next? Look up the biggest Android fork in China and write pointless long article about it how it’ll take over Google’s Android.
HarmonyOS and Excel comments are mixed up.
HarmonyOS is not an open-source OS, only partially components.
P.S.
What is happening with these comments from other articles?
this is funny. it looks like the same base article, comments and all has been repurposed and edited or something for 3 (or more) different articles.
started off as something for some office suite… and then it was some antivirus thing.. and now it’s some iffy os.
wtf is that writer doing? no one reads or comments on the ad articles and he has to reuse old ones to make it look like it’s getting traction? (though it shows up as 0 comments on the homepage, so that can’t be it?)
“HarmonyOS does not have Google. Huawei’s HarmonyOS is a proprietary operating system..”
vs
“HarmonyOS is open-source, which means that it can be customized by developers and manufacturers.”
Both are quotes from the article. So which one is it, open source or proprietary?
Without windows-based videogames able to run, it’ll stay niche.
Huawei better put serious money into a harmonyOS port of Wine, DXVK or Proton if it wants its machine being more than web browsing thin clients.
I won’t trust a foreign OS.
Deeply foreign, I meant. In so many ways.
“I won’t trust a foreign OS.”
You trust Microsoft Windows, Google Android and Apple operating systems just because they are from businesses in your country ? Talk about blind nationalism.
Do not forget also that there is a world outside of USA and that for most of human beings, your favorite operating systems are also of a foreign origin, and as hostile towards them as they are towards yourself.
HAHAHAHA – did you really say “it could pose a serious threat to Microsoft’s Windows operating system”?
It won’t run programs or real games, so it will be useless.
I mean, there are some people who apparently love using tablets and phones for everything, but mostly are people who will barely do anything with their brain in life.
I mean, even if Photoshop, and others are available for iPad, do people think iPad is a threat for windows? not really, maybe for useless consumers who will just, like with a phone, be happy and move on, but not for professional industries which are the ones who matter the most, because are the ones who generate more revenue, since they buy the most expensive hardware and software.
Nothing consumer computing related really makes much money, unless it is data from users that get sold for AI or Ads.
Who knows where this comment is going to wind up. It’s in response to the article about Huawei’s HarmonyOS (HOS) operating system.
Two places in the article it says HOS is open-source. One place the article says HOS is proprietary. Uh, I’m pretty sure it can’t be both. Which is it? If there’s some fine distinction, somebody needs to explain it.
After all of these issues with Ghacks articles and misplaced postings, I’m reminded of this
https://www.ghacks.net/2019/10/07/ghacks-has-a-new-owner-and-that-is-a-good-for-everyone/
@vanp,
Note: I replied to you on September 6, 2023 at Around 2:20 pm, but it was still remain blocked after more than half a day, so I replaced the quoted URI scheme: https:// with “>>” and reposted.
The current ghacks.net is owned by “Softonic International S.A.” (sold by Martin in October 2019), and due to the fate of M&A, ghacks.net has changed in quality.
>> ghacks.net/2023/09/02/microsoft-is-removing-wordpad-from-windows/#comment-4573130
Many Authors of bloggers and advertisers certified by Softonic have joined the site, and the site is full of articles aimed at advertising and clickbait.
>> ghacks.net/2023/08/31/in-windows-11-the-line-between-legitimate-and-adware-becomes-increasingly-blurred/#comment-4573117
As it stands, except for articles by Martin Brinkmann and Ashwin, they are low quality, unhelpful, and even vicious. It is better not to read those articles.
>> ghacks.net/2023/09/01/windows-11-development-overview-of-the-august-2023-changes/#comment-4573033
By the way, if you use an RSS reader, you can track exactly where your comments are (I’m an iPad user, so I use “Feedly Classic”, but for Windows I prefer the desktop app “RSS Guard”).
RSS Guard: Feed reader which supports RSS/ATOM/JSON and many web-based feed services.
>> github.com/martinrotter/rssguard#readme
Regarding “Huawei’s HarmonyOS” you asked about, the developer has stated that it is planning to open source, however the actual situation has been shelved (no such movement).
HarmonyOS – Wikipedia
>> en.wikipedia.org/wiki/HarmonyOS
Therefore, it is “proprietary software”.
Moreover, both the company and its production base are in China. China, Russia, Israel, etc. are “Authoritarian state” and products and companies based in those countries are under state control, and privacy policies can become “famous and innocent.” Those products should be avoided.
Correction of sentence
Before correction: “famous and innocent.”
After correction: “nominal name only titular.”
owl, thanks again for the great info.
HarmonyOS doesn’t run Windows apps. It is no threat to Windows.
I agree.. i bet it cant even run wallpaper engine, it probably has the worst compatibility with software.
iam vary satisfied this work
http://crackscoop.com
ah, wonderful, this message/article cross-posting hasn’t been fixed.
Ignore my comments.