Read articles behind paywalls by masquerading as Googlebot

Martin Brinkmann
Feb 26, 2016
Updated • Feb 11, 2020
Internet
|
68

The Internet is at a tipping point. The continued rise of adblocking has put an end to the revenue model that relies solely on ad dollars to operate websites and businesses.

Especially news sites have started to experiment with ways to diversify income sources, and one prominent option that sites like The Wall Street Journal, Financial Times, The New York Times, the Times, or The Washington Post have implemented or tested is the paywall system.

There are different types of paywalls but they all have in common that they block access to content; this may happen directly when the first article is opened, after a certain number of articles have been read on site, or as an excerpt system that displays the first paragraph to the reader and below that sign-up information to read the rest.

Paywalls may not always require users to pay money for access. Some sites may require users to sign-up to use the site but won't charger users once they have signed up.

news site paywall

It may make sense from a business point of view, and may be more lucrative than battling it out with users who run adblockers, but there is a downside to it both for the paywalled site and the blocked user.

Sites lose a high percentage of visitors if they implement a paywall system. It is unclear how high the percentage really is, and it probably varies from site to site, but it is likely a lot higher than the percentage of visitors who subscribe to the site after being presented with the choice to subscribe to read the desired article.

For users, it can be really frustrating to follow a link to an interesting sounding article just to be blocked from reading it once the resource has loaded; it is a waste of time for many, especially if no content is provided prior to signing up or subscribing.

Masquerade your browser

It is no secret that news sites allow access to news aggregators and search engines. If you check Google News or Search for instance, you will find articles from sites with paywalls listed there.

In the past, news sites allowed access to visitors coming from major news aggregators such as Reddit, Digg or Slashdot, but that practice seems to be as good as dead nowadays. Some may still allow it but it is trial and error, and the workaround may be shut down at any time.

Another trick, to paste the article title into a search engine to read the cached story on it directly, does not seem to work properly anymore as well as articles on sites with paywalls are not usually cached anymore.

Tip: check out the following add-on that you may use to bypass paywalls:

User-Agent and Referrer

You are probably wondering how sites block or allow access to the site's content. The methods have have improved over the years, and it is no longer enough to simply change the referrer of the browser to https://www.google.com/ to gain full access to a site's content.

Instead, sites use various checks that include user-agent, referrer and cookies, and sometimes even more than that, to determine the legitimacy of access.

General information

Probably the best way to masquerade the browser is to make it appear to be Googlebot.

  • Referrer:  https://www.google.com/
  • User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html

Note that the option does not work anymore on may sites. It may be better to try and masquerade as coming from Twitter or other social media sites.

Firefox

referrer

Firefox users need two browser add-ons for that: the first, RefControl, to change the referrer value when visiting news sites, the second, User Agent Switcher, to change the user agent of the browser.

Update: RefControl is no longer available. You may try this alternative instead. End

  1. Download and install both extensions in the Firefox web browser.
  2. Tap on the Alt-key, and select Tools > RefControl Options.
  3. Click on "add site", enter a domain name under site, select custom action, and enter https://www.google.com/ as the referrer.
  4. Repeat this for all news sites you want to access (some may not work even if you make the changes, so keep that in mind).
  5. When you are done, close the configuration window.
  6. Tap on the Alt-key again, and select Tools > Default User Agent > Edit User Agents from the menu.
  7. Select New > User Agent, and replace the string in the User Agent field with Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html). Name it Googlebot.
  8. Exit the menu.
  9. Before you access these sites, tap on Alt, and select Default User Agent > Googlebot.

This is all there is to it. It is a bit unfortunate that there is no extension for Firefox that changes the user agent automatically based on the sites you visit.

Google Chrome

Google Chrome users can install extensions like User Agent Switcher and Referer Control that are available for the browser to do the same.

There is however another possibility, and that is to create a custom extension which automates the process in the browser.

Instructions are provided on Elaineou. All it takes, basically, is to create a new directory on the local computer, create the two files background.js and manifest.json inside it, and copy and paste the code found on the site into the files.

You need to enable "developer mode" on chrome://extensions/, and can then select "load unpacked extension" to pick the folder you have created the two files in to load the extension in Chrome.

You may modify the list of sites it supports to add new ones.

Summary
Read articles behind paywalls by masquerading as Googlebot
Article Name
Read articles behind paywalls by masquerading as Googlebot
Description
Paywalls prevent Internet users from reading one or more than a handful of articles on news sites. Find out how to access these articles nevertheless.
Author
Publisher
Ghacks Technology News
Logo
Advertisement

Tutorials & Tips


Previous Post: «
Next Post: «

Comments

  1. Frits said on October 26, 2018 at 6:56 pm
    Reply

    Like which ones ?

  2. Anonymous said on October 3, 2018 at 11:28 pm
    Reply

    There are better methods to bypass walls now.

    1. Frits said on October 26, 2018 at 6:55 pm
      Reply

      So as ?

  3. xavest@gmail.com said on February 24, 2018 at 4:16 pm
    Reply

    RefControl does not seem to be supported by Firefox any longer. Does anyone know how to do the same with Referer Control (not the same add on as the later), and User-Agent Switcher?

  4. Ronaldo said on October 25, 2017 at 9:27 am
    Reply

    Just my 2 cents here. This webbrowser seems to bypass most of the major newspaper paywalls straight off the bat. Just tried it, LATimes, Chicago Tribune and even NYTimes seems to work.. amazing stuff. Not sure if this was the intended use though, but it works :)
    https://itunes.apple.com/us/app/web-captain/id1296674405

  5. jg said on August 1, 2017 at 8:45 pm
    Reply

    SortingHat
    agreed

  6. SortingHat said on June 10, 2017 at 5:11 pm
    Reply

    When the internet was more newish the world leaders and elite have not any need or care for it. Now that governments have been overthrown via the use of smartphones they are suddenly paranoid if it can happen there it can happen here!

    Sadly they are focusing on invisible terrorists which are just ordinary citizens and ignoring the real flags and go thru the whole *right wing Christian terrorist* bull shit witch hunt. The web is pretty much locked down now and the only people who use it are those who shop and chat. Google uses Watson AI (the dumb version) that just generalizes popular searches and gives you what it thinks people want the most and that will be anything related to *shop and chat* if you search for anything outside of that you will just confuse it further giving you weird results.

    No it’s not you being an idiot. It’s not your keyboard acting up and clearing your cache won’t fix a damn thing. The fault lies in the algorithm itself.

    The internet is shopping channel 2.0 now and if you type in research stuff your a freak that deserves to live in a cupboard under the stairs. At least that’s the minds of the super rich and wealthy. The government all they think is your the next Jihad terrorist or right wing terrorist or upper wing terrorist or any kind of wing terrorist.

  7. SortingHat said on February 27, 2017 at 12:32 am
    Reply

    Most websites are mobile only now like YouTube if you put your computer to sleep and turn it back on you briefly see the mobile version of YouTube and it looks all different then a minute later it reverts itself back to desktop mode.

    On Mobile mode you can’t see the comments unless you use the touch sensitive pad to scroll way down and on a PC phone oriented sites look ugly and hard to navigate.

    The reason sites use the slide show format for everything is because it is set with you using your finger to swish thru them and on a tiny screen you only see one frame at a time where on a DESKTOP or even a laptop you will see the whole thing. It comes across as awkward.

  8. SortingHat said on February 27, 2017 at 12:30 am
    Reply

    Since the smartphone industry has come about in the mid 2000s the global elites have decided that you as an individual have no rights to own media and not only not own media but not even own the switch to turn it on!

    Smartphones may sound cheaper then a computer but you pay thru th3e nose for every stupid thing and the costs escalate very quickly where on a PC at least it used to be you just pay for your internet connection and not much else as you wouldn’t share personal info online and you could access your media anytime you wanted and share it sometimes up to 3 devices or even unlimited like how Amazon Mp3s are.

    Now it’s all different to train young millennials to accept a one world system where it will be in your forehead/forearm and to put up or shut up. Nope not going to work here!

    BTW: I’m typing on a DESKTOP computer not a mobile phone. :) I got a keyboard and mouse but most websites are designed with touch screen in mind which is why we need Ad Block or the whole web would be totally jumpy and unusable not just the site you go to.

  9. Bri said on February 19, 2017 at 3:17 am
    Reply

    Thank you :)

  10. PB said on February 12, 2017 at 9:53 am
    Reply

    If websites asked for a reasonable fee to access their articles most people would take up the offer. The problem is that they are much too greedy.

    1. SortingHat said on June 10, 2017 at 5:13 pm
      Reply

      I agree 20 dollars a month will take away paying rent/utilities.etc. Paywalls only those with good jobs can afford it and that is an ever shrinking few.

  11. JohnB said on January 19, 2017 at 4:57 pm
    Reply

    Brilliant. Thank you. For me, this method allows Firefox to bypass the paywall on most major news sites.

    “It is a bit unfortunate that there is no extension for Firefox that changes the user agent automatically based on the sites you visit.”

    Actually, contrary to what the article states, there *is* an extension called UAControl that will automatically set the User-Agent on a per-site basis:
    https://addons.mozilla.org/en-us/firefox/addon/uacontrol/

    Just install this extension and configure it to use the “Googlebot” user agent settings for each news site you want to access.

    Voila! – you no longer need to manually set the User-Agent every time you want to access a news site.

  12. Klaus said on September 24, 2016 at 8:47 am
    Reply

    Before I go to all the lengths downloading and implementing what you described above, do you know whether it works on Sueddeutsche Zeitung? They still have enough ads on their sites. I just don’t see why they have a paywall. They won’t make enough money to rescue their ailing, but otherwise excellent, newspaper.

  13. Lenny Schafer said on September 7, 2016 at 11:41 pm
    Reply

    I would be willing to pay a yearly reasonable flat fee that would allow me to enter any paywall; granting me a total number of entries however I would like to spend them. This is what I call a “seasons pass” model. This is a business waiting to happen as the paywall thing grows. Another small business project would be the creation of a paywall filter app that just automatically IDs and blocks such listings on news aggregators. Maybe these things already exist; anyone reading this let me know.
    All the workarounds I have found so far are complicated and annoying.

  14. RmR said on July 13, 2016 at 12:54 pm
    Reply

    When it says to create new directories and files, are those text files and folders respectively?

  15. kalmly said on May 16, 2016 at 4:36 pm
    Reply

    I am very late to this discussion, and I don’t have anything helpful to add to it. I do want to say this: Sadly, I had to turn my adblocker back on for this (Martin’s) site because my browser was eating up resources to the point it took forever to open a page when I clicked a link. I tried stopping the videos on the page, but that didn’t help much. When FF began to hang and crash, it was too much to put up with.

    1. SortingHat said on February 27, 2017 at 12:34 am
      Reply

      It’s because it thinks you are using a touch device. If you aren’t using one it will slow everything way down as resources are being used up to try to make your PC into *Phone* mode. Not to mention spy stuff both from advertising and government crap will make your CPU go almost to 100% unless you have Don’t Track Me turned on.

  16. Paula said on May 15, 2016 at 12:22 pm
    Reply

    Unfortunately doesn’t work anymore for most of the sites.

  17. Big Maq said on March 8, 2016 at 8:10 pm
    Reply

    The Elaineou solution in Chrome works for the front page and major sections of the WS Journal.

    However, when checking a specific article, got the same paywall issue.

    Am I missing something?

  18. br33ch said on March 5, 2016 at 10:23 pm
    Reply

    I just need to mention that this should not be left on permanently because some sites may be configured with a firewall that can detect it. WordPress running WordFence for example allows blocking from 1 minute to 1 month.

  19. Rotten Scoundrel said on February 27, 2016 at 5:10 pm
    Reply

    The rather obvious thing, well to me, is that the “news” in question is available for about a zillion other sites that are scraping the likes of the WSG et al. So why bother getting through a paywall?.

    Use google for a search if you must, but then use something like dogpile (or any other search aggregators) for drilling down on a story you want to read in full for free. :)

    https://en.wikipedia.org/wiki/News_aggregator

  20. Mick said on February 26, 2016 at 8:51 pm
    Reply

    I use a variety of means to get around paywalls, depending on the site in question. The extension Paywall-Pass might still work from time to time, in other cases it’s getting there from the Google search for the title, yet other cases might call for turning on private mode, and sometimes it’s just enough to turn off JavaScript on a particular page to get read of the blocking ad.

  21. beach boui said on February 26, 2016 at 6:37 pm
    Reply

    Internet advertising purveyors are killing themselves. The loud, flashy, attention grabbing ads are more than annoying. They harder they try to grab our attention, the more determined we become to avoid the noise, both visual and audible. The public will accept advertising that doesn’t hurt their eyes or insult their intelligence. The problem is not with the users, or the sites that allow third-party advertising. The problem is with the companies that create and place the third-party advertising.

    1. SortingHat said on February 27, 2017 at 12:36 am
      Reply

      It’s because they are like roaches when we shine the light on them they freak out about it. It’s a sign our broken capitalist/monopoly system is a cancer and we are seeing the early stages of monetary cancer in our society.

  22. mkdante381 said on February 26, 2016 at 6:31 pm
    Reply

    For noobs!! Dear author pls change link to https://addons.mozilla.org/firefox/addon/refcontrol/ and browser automatically redirect to your language…

  23. oz said on February 26, 2016 at 4:40 pm
    Reply

    Good information presented here, so thanks for that and for being so “open-oriented” in all your other articles. I do block ads and a few other things on this site, but I did make a donation a few months back, so hopefully that evens things up a bit. I generally don’t take the extra time to sign in, but do check for tips like this several times each day, and do post as a guest now and then.

    Thanks again for everything.

  24. Jed said on February 26, 2016 at 3:22 pm
    Reply

    I tend to avoid paywalled sites and look for the same information elsewhere, their loss. I also use an adblocker and block trackers with no qualms. If it was a couple of text ads along the side with maybe a small picture like you get in newspapers then I wouldn’t mind, but instead there’s moving ones, ones that autoplay videos, and lots of large, in your face ads etc that detract from the website experience.

  25. Jeffrey said on February 26, 2016 at 2:02 pm
    Reply

    Martin

    Something that has worked for me on some sites it to use Chrome’s “Open Link in Incognito Mode”. Granted this will not be effective everywhere, but it helps me when I go over the NY Times’ limit of 10 articles per month.

    1. Martin Brinkmann said on February 26, 2016 at 2:13 pm
      Reply

      You could also delete cookies in normal mode, but private browsing mode works fine as well on sites that rely solely on cookies.

  26. Henk van Setten said on February 26, 2016 at 12:46 pm
    Reply

    There is a huge moral difference between using ad blockers, and using tricks to bypass a paywall. I feel the first is morally acceptable, while the second is not acceptable at all.

    Using an adblocker means refusing the “gift” of some advertisements that a publisher wants to push to your device without you asking for them. Put simply, of course you’re quite free to refuse anything you didn’t ask for in the first place. No problem at all (except the publishers’ need for a better ads strategy).

    Using tricks to bypass a paywall is a very different thing, though. It means you are sneakily using backdoors to get a product that you’re supposed to pay for, without paying for it. In plain old language we have a simple word for that: it’s called stealing. Or shoplifting.

    So I must say I’m really disappointed that in this case, Martin Brinkmann degraded his usually excellent blog by telling people where to get the best lock picking tools. Stealing is unacceptable; but promoting burglar tools isn’t very acceptable either.

    Just my opinion, of course.

    1. Martin Brinkmann said on February 26, 2016 at 1:19 pm
      Reply

      As a webmaster, I have to disagree. I understand that this is a controversial topic but from a revenue point of view, it makes no difference if people are using adblockers or bypassing paywalls.

      1. Rocky said on February 26, 2016 at 7:56 pm
        Reply

        I agree Martin that from a Revenue point of view there is no difference between adblockers and bypassing paywalls There is a big difference though between blocking ads that are presented to you and bypassing paywalls by masquerading/pretending to be somebody other than yourself.

        I prefer if I can to leave ads to display for the simple reason that content providers have to be remunerated in some way – I can envisage blocking some disruptive or indeed unacceptable ads ( I am constantly getting ads about Asian ( or other ) ladies which I find to be objectionable ads – no idea why these ads present as my web browsing is not in any way towards anything “iffy”) . I could not see myself though attempting to bypass paywalls in the manner envisaged in this article.

        If websites cannot find some way to temper the more extreme ad providers then the Web could indeed be at a tipping point.

      2. DVD Rambo said on February 26, 2016 at 4:18 pm
        Reply

        Martin,

        I like to turn off adblocking on useful sites like yours. However, with Ghacks allowing video ads (two running, currently from Sekindo) to eat into my limited bandwidth, I will now use adblockers all the time. I think it would be a public service if you no longer allow video ads, and move to static ads only. I’ll PayPal you $10 to help with your site. Should I use news@ghacks.com to send the money. I will not use GoFundMe as they have spammed me in the past and are now blacklisted on my mail server. Thanks for your excellent site.

      3. Martin Brinkmann said on February 26, 2016 at 4:25 pm
        Reply

        Thanks for your support. You can use the PayPal link on our support page for that: https://www.ghacks.net/support/

    2. Tom Hawack said on February 26, 2016 at 1:08 pm
      Reply

      The difference is not plain semantics and I agree with the terms of this distinction. But I do not believe morality is concerned when the link between information and promotion is by itself a debate, especially on the Web. Are search engines promoting terrorism, pornography, hatred when they include links to sites that do? My opinion is that morality is a very personal matter and becomes proselytism when served as an argument in the pertinence of informing or not. Heck! Would be promotion a developed argument incentivizing a given practice, and as far as I understand it information is not an argument by itself, unless to consider that a reader is totally irresponsible, acknowledging that such a consideration is the basis of demagogy and dictatorship.

  27. Rocky said on February 26, 2016 at 10:54 am
    Reply

    Martin,
    I find the content of this article to be very close to crossing a line on fairness/fairplay. If a site chooses to implement a pay model then seeking to bypass that model in order to access content sits uncomfortably with me . I agree that the ads can be disconcerting and I wish that some sort of accepted practice could be developed in that regard but as Tom Hawack says “If I were committed to a news source (or to any other genre) I’d consider paying for it”

    1. Yuliya said on February 26, 2016 at 1:10 pm
      Reply

      Not necessarily. You may come from a Google search to a website that is using this model, yet you have never visited that website and you may also have no intentions to do so in the future. It’s either let me see the content that I’m looking for or get out of Google’s results list. Otherwise it’s just a waste of my time and my bandwidth. Not only that, but you’re only being offered a weekly/monthly plan. I just want one article, or rather one paragraph from it.

      Make it obvious so I don’t click on that result at all, at least. Otherwise, once the page is in my PC one way or another I feel that I am allowed to alter its content in any way that I can. (from that screenshot it looks like the kind of website that loads the whole page in background but it restricts your view with an overlay)

      1. Pants said on February 27, 2016 at 3:36 am
        Reply

        ^^ This.

        Google (although its not a problem of their making) should mark paywalled sites as that. I don’t begrudge a site deciding on a paywall – that is their decision. I also do not begrudge sites getting listed on google search, as it’s rather essential to drive traffic. But they know this and use it to allow access and either show a snippet to TEASE you, or use asshat crap like Wired (it’s not paywalled) where slightly into the article an overlay will come up and block everything (easily bypassed, BTW).

        I don’t begrudge a site using ads (but I do hate ad networks that have no interest in advert control – eg bullshit flash/animation/letting malware thru/not forcing https/audio/autoplaying etc etc) but I do not like sites that use too many ads or use asshat things such as adverts following around the page.

        I also don’t begrudge the tracking, if that makes adverts more effective. I would rather there were less ads because they are more effective. As long as I, as an individual, can block the tracking, I don’t care.

        This article is about paywalls, but ties into advertising models. The whack-a-mole game of adblockers and adblock killers, and adblock killer blockers, and anti adblock killer blockers .. ad nausuem will continue. It’s really starting to heat up and I can see the point where most major sites will PUSH data, it’ll all be server-side – eg, IF your browser has accepted a cookie, and IF your browser has allowed connection to certain third parties, and IF your browser has downloaded the adverts (say, into local dom storage), then, we will push you the article text and images. Whether you allow your browser to display the ads is irrelevant. The website has done it’s job in serving an ad and will be paid.

    2. Martin Brinkmann said on February 26, 2016 at 12:45 pm
      Reply

      Rocky, I agree with you 100% that paying (or displaying ads) is the honorable thing to do if you access a site’s content regularly. I think it is important that content producers get paid for their work.

      1. Rotten Scoundrel said on February 27, 2016 at 5:05 pm
        Reply

        Straying OT here. :)
        Martin, in a fit of compassion for your monetizing the site, which I fully understand and agree with, I decided to free up your site from adblockers in my browser. So, I commanded UBlock to accept all on this site now and for the future. The same for Adblock and refreshed the page. I got a single picture ad, so I thought this ain’t that bad.

        But then I noticed, “Advertisement” and large blank spaces sprinkled down the right side of the page where something else was obviously blocking those ads. I checked noscript and there are about 10 other sites that I needed to allow. Including things like Font managers. Those sites offering fonts do so, so they can get a taste of the tracking dollars too. It is not like there not enough free fonts in the world already that a site needs to use a specific Font for something.

        And, there’s the rub. It is not all about unblocking your ads, it is about unblocking my browser to all sites who advertise on your site. So for now, I am OK with unlocking ublock and adblock, but I’ll be damned if I will unlock my browser to all the others.

        To see a most egregious example of hangers-on go to the Fox-site for Hell’s Kitchen videos. Turn off noscript then when the video runs, check it for how many sites are in that list. I stopped counting at 37.

        So, to me, it is up to the advertisers to play fair if they want me to open my browser for them. It then falls to the webmasters of the sites I visit that want me to then open my browser, to use ads that follow a set of moral and ethical rules.

      2. Martin Brinkmann said on February 27, 2016 at 5:35 pm
        Reply

        From a user perspective, that’s bad and I know that. I keep away the nasty ads and stuff here on Ghacks for the most part, but there is only so much that I can do about it from my end.

        The situation would be better with a dedicated sales team or enough pull to attract premium advertisers (still waiting for that Microsoft email ;P).

    3. Tom Hawack said on February 26, 2016 at 11:18 am
      Reply

      @Rocky, and before letting Martin reply for himself, I just wish to bring this precision since you’ve mentioned my comment : my statement regarding paying for a site I’m committed to is motivated by practical reasons (the hassle of getting two add-ons or bringing up a dedicated script scheme) rather than by an ethical approach, unfortunately. Not that I’m an evil person but because of the easiness — when applicable — of getting things for free when it’s not too much of a work. To W(w)ome it may concern : forgive me!

  28. Jojo said on February 26, 2016 at 10:33 am
    Reply

    There is a FF extension called Paywall Pass. Used to work like a charm but lately doesn’t work at all. But maybe something to do with my computer or that I switched to FF ESR not too long ago. So perhaps it will work for others.

  29. Tom Hawack said on February 26, 2016 at 10:22 am
    Reply

    Especially news sites are concerned indeed, whatever the country perhaps; here in France all the same.

    If plain news can be accessed anywhere it is of course the editorials, the news analysis, which are the loss when not partially/totally accessible because of the sites’ policy.

    I won’t go through the hassle of installing two add-ons, of which one having to be manually triggered each time, in order to read more. If I were committed to a news source (or to any other genre) I’d consider paying for it. No such commitment at this time though.

    What I don’t understand is that the percentage of guests on a news site (or other) which do not use an ad-blocker doesn’t seem to allow a required income for those sites. Is it so that ad-blocking is becoming so intensive that the natural balance between ad accepted and ad refused is no longer a sufficient ratio to allow sites to live or is it that their profit is destabilized? I always thought that there would be enough users accepting ads to balance correctly those refusing it.

    I don’t want to compare what is not comparable yet I’ll dare this : you really have to be very fond of a lady to not search elsewhere when she seems far too demanding.

    1. Yuliya said on February 26, 2016 at 1:18 pm
      Reply

      “that the percentage of guests on a news site (or other) which do not use an ad-blocker ”

      I think this could be roughly calculated by the number of users that a browser has and the number of users the more popular ad blocking extensions have. Few years ago I remember the number of users who had an ad blocking extension installed (or any extensions at all) was quite low. Things might have changed though, I noticed even PCs at my college had ABP for Firefox installed.

      1. Tom Hawack said on February 27, 2016 at 7:29 pm
        Reply

        @Corky, I believe so as well. I’m convinced that a wide majority of anti-advertisement users don’t hold this position for a whatever anti-system demagogy reason. Advertisement in itself is not the problem from there on, I believe bringing to others’ attention your work, your product is a natural component of mankind, at least within the trade scheme.

        Now, if advertisers calmed down a bit, stopped being hysterical and started thinking on a long-term, they’d realize that there is one and one only alternative to keep their business flourishing : less ads, better ads, advertisement police to break malvertisement (Web specific of course). The customers would spend less on ads, those ads bothering less the consumer (or not at all, who knows?!), they would be more productive. In this scenario everyone wins except the ad business in terms of income : there is a too big part of the cake swallowed by the ad business, and this is a fundamental, root problem.

      2. Corky said on February 27, 2016 at 6:49 pm
        Reply

        @Tom Hawack, IMO the amount of people blocking adds would drop if 1) They were less intrusive (audio, obscuring content) and 2) They didn’t crash or lockup people browsers (not responding, plugincontainer.exe using all available RAM (4GB+))

      3. Tom Hawack said on February 26, 2016 at 2:27 pm
        Reply

        @Yuliya, I have friends who are advocating another way of dealing with the problem of Web sites existence. Up to now I had to disagree but I’m starting to wonder if they are not right when they state that individuals/companies that make it right are those which don’t depend on their sites for a living, those which have a site as a window to their business which is elsewhere. News sites are moving from paper to digital supports, they have no other income and ads is essential and therefor anti-adblocking as well…. we’re likely to assist to the disappearance of many sites which have no other income than the ads, unless they manage a faithful audience of subscribers ready to spare a few bucks, so many sites requiring low fees. Anyway one thing is sure : you don’t make business from a website alone, if so you struggle to survive.

      4. Tom Hawack said on February 26, 2016 at 2:08 pm
        Reply

        Martin, 40+% of users blocking ads would be less a problem if advertisers shared income in a more fair way.
        The users are in between an advertisement terrorism and the acknowledgement that sites indeed have to make their way. Of course it is possible to make per-site exceptions (tough nevertheless when the browser’s ad-blocker is backed-up my system-wide blockers) but the problematic is elsewhere, and I believe this elsewhere is not in advertisement as such but in the very policy of the advertisement policy : 1- No honest deal with the “vectors” when the share is balony, 2- No honest deal with the users (the consumers) when malvertisement is not eradicated, when the very philosophy of the business is to force and not seduce, to try to enforce by all means delivered ads after imported tracking, but never (even if a few voices are heard here and there) to promote better quality, less quantity in the scope of respect. Not only respect when a new way of advertising would be profitable to all… except perhaps to the advertisers themselves.

        it’s a true problem, but I will not accept ads as they are now and will always prefer to pay a website without any intermediaries which make a living in the old import-export scheme, that is buying/selling comfortably installed and emphasizing on the margins, that of differentials : the least they could do is reconsider their attitude. They want war and they have war, what more to say?

      5. Martin Brinkmann said on February 26, 2016 at 1:26 pm
        Reply

        Most sites report between 10% to 20%, but on tech sites like mine, it is more than 40%. In my opinion, sites will find new ways to adapt to the situation or die, and it will not be pretty for Internet users as content will get worse.

        I expect to see more advertorial content and the pushing of affiliate links directly in content. It is already happening, and morally less stable sites may have no issues pushing stuff that sells in articles, and even write articles around these directly.

        I’m still trying to figure out how to survive the next 30 or so years.

  30. Angry Thinker said on February 26, 2016 at 10:03 am
    Reply

    Thanks Martin :-)

    1. CHEF-KOCH said on February 26, 2016 at 10:27 am
      Reply

      :-)

  31. yoav said on February 26, 2016 at 9:33 am
    Reply

    The problem with the paywalls is that you can’t share anything you read or refer people to the article, because you can never be sure who has access. So for me bypassing them is besides the point.

    1. Rick said on February 26, 2016 at 6:08 pm
      Reply

      There is a wonderfully fantastic firefox addon:

      http://maf.mozdev.org/

      It allows you to save an entire page in a single file.

      1. dingoloid said on December 5, 2017 at 5:12 pm
        Reply

        A better firefox plugin is personal blocklist you add POS paywall sites to the list and they never show up in Google search results again.

      2. Dave said on February 27, 2016 at 10:33 am
        Reply

        Personally I prefer UnMHT which saves excellent MHT files.

    2. Jojo said on February 26, 2016 at 10:30 am
      Reply

      Just copy the whole article using something like Pocket or to the clipboard and then save it somewhere. Not that complicated….

  32. Shivraman said on February 26, 2016 at 9:24 am
    Reply

    Thanks for educating us readers about AdBlockers first and now how to bypass paywalls.

  33. Tmue said on February 26, 2016 at 9:05 am
    Reply

    This seems to be the correct link toRefControl:

    https://addons.mozilla.org/en-US/firefox/addon/refcontrol/

  34. Owl said on February 26, 2016 at 8:43 am
    Reply

    Peope use adblockers because of tracking, malvertizing and sleazy ads. I don’t know what the technical issues are, but why don’t sites go to static banner ads? On tech sites, ads for tech things, etc. No need to block ads then.

    1. Decent60 said on February 26, 2016 at 5:18 pm
      Reply

      Some sites sell ad space but it’s less $$ in the long run (not to mention more time consuming) than having a script that pulls information and relative ads based upon what’s in the user’s cache.

      Personally, I don’t mind the Tracking so much, but what I hate (and what originally got me started using adblockers) is ads that ruin the page layout (WordPress operated sites are what started it for me) and now ads that play audio instantly, even if I have such things turned off in my browser.
      I used to listen to music on my home theater system attached to my computer (or even when I use headphones), it gets very annoying to have an ad that plays louder than my music to come through with ” XXXX can help treat your erectile dysfunction”. My neighbors actually complained about that ad rather than the music.

      1. Martin Brinkmann said on February 26, 2016 at 5:27 pm
        Reply

        I’d love to host all ads on my server and deliver them as static images or text, but this is not feasible for many reasons (time, getting advertisers to buy directly from you, hosting your own ad server, managing campaigns, payouts…).

        I’d love to see a company create a service that would provide only these types of ads. I could see the operators of Startpage or DuckDuckGo start such a service, as they have experience in running ads without tracking users and probably also the resources to create one.

  35. Rick said on February 26, 2016 at 8:41 am
    Reply

    Martin, your link to refcontrol is wrong. It is pointed at Referrer Control that does not allow site specific referrals from what I can see.

    1. Martin Brinkmann said on February 26, 2016 at 9:16 am
      Reply

      Thanks, corrected the link.

      1. tim said on October 16, 2018 at 10:36 pm
        Reply

        nice and easy way to get peoples facebooks bro

      2. Anonymous said on November 13, 2018 at 11:39 am
        Reply

        Is this true?

  36. Jimmy James said on February 26, 2016 at 7:47 am
    Reply

    Well if they were subtle with their ad placements and respectfully asked people to turn off their addblocker for their site, then the system wouldn’t be broken. Like the ads on Ghacks are not invasive and don’t bother me at all :)

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

We love comments and welcome thoughtful and civilized discussion. Rudeness and personal attacks will not be tolerated. Please stay on-topic.
Please note that your comment may not appear immediately after you post it.