More Introductions, 2024 So Far & Exciting Things To Come

We’re only 11 weeks into 2024 and already so much has happened. Sorry for being so quiet, I’d have liked to say more over Twitter, but some things have needed to stay hush, while other things were time consuming but not all that interesting.

Firstly - the secret stuff - I took a 2 week family holiday to Thailand in February. We spent a week on the beach and a week in Bangkok learning two major things - Thailand’s winter is HOT and the food is out of this world. Also, I proposed to my partner on the beach on our anniversary and she said yes! So I’m now officially engaged.
This trip needed full OPSEC due to all the DDOS problems we’ve been having, I didn’t want Rentry to be a major target while I was away. Luckily we had just 1 minor incident halfway through, with a major one on the day I returned, which led to the tweet about Cloudflare.

Secondly - introductions - I’m pleased to introduce Sarah, the new community support manager! Sarah is a childhood friend of mine and I couldn’t have found a better person for this role - she’s exceptionally hard working, caring and unphased by anything. Choosing someone to trust with control over Rentry URLs wasn’t easy, but I have absolute faith that Sarah is deserving of that trust and that she will make honest and fair decisions over any issue that she deals with.
As I’m sure many of you would like to know more about her: Sarah is 29, lives in England and has a Staffordshire Terrier called Breeze.
She’ll be helping out with answering your issues over email and Twitter DMs, handling URL claim requests and keeping Rentry safe by reviewing pages that get reported or flagged as dangerous.

This Year so Far

Load Speed

2024 started with Rentry’s average page load speed plummeting, especially at certain times of day. There’d be times when each page took over 10 seconds to load, which is crazy high for a modern website and way above the 0.5s goal. Investigating, I figured out the problem was a mixture of a few things:

The new upgrades brought in at the end of last year were taking more processing power to produce a page.
We had (in November) switched from saving the HTML of the pages into the main database on creation to now processing them live on each page load, as this made the upgrades that followed possible (as otherwise every time we add anything new to the markdown system, a full re-process of every post would be required). Of course this puts a major new burden on the live processing ability.
There was a bug in how some pages were processed - for example the table processing system was causing some pages to take 60+ seconds to load, even if they didn’t have any tables on them.
Rentry was getting relentlessly scraped, particularly as the automatically given URL size was set to 5 characters, which meant that scrapers were simply trying every possibility and on average landing a result every 5 requests.
The long loading times were causing a compounding issue, where the number of requests was exceeding the number of server processes able to handle a request.

As mentioned in a Tweet, the issues are now completely fixed and this sets Rentry up nicely for further upgrades in the future. To fix these problems, these solutions were implemented:

Creating a new two layer caching system, saving each page’s HTML to memory for a short time and to a file cache for a long time. (Previously it was just the memory cache).
Fixing the processing bugs.
Adding in pre-processing look-aheads, for example checking if any tables exist by searching for the symbols required to create them. Any markdown extensions not needed are then disabled for that page.
Adding a processing time-out, if a page takes too long then an error is shown along with the /raw/ version of the page.
Creating a new fail2ban module which identifies scrapers and blocks their IP.
Increasing the number of characters in an automatic URL to 8
Switching to a far more powerful dedicated server.
Adding a huge number of server processes.

DDOS & Cloudflare

With the constant and still unexplained DDOS attacks taking the shiny new Rentry main server offline, Cloudflare has become the only option to keeping the service available 24/7. After numerous tests to ensure that it doesn’t overly cache pages after edits have been made, Rentry.co was fully integrated with Cloudflare and is now under its protection. We’ve had one major attack since then which led to a total of 1 minute of downtime, with lessons learned. Next time we’ll be even better prepared.

URL Claims

We’ve now officially hit 16,000 URLs claimed and processed! I (and more recently, Sarah) have been keeping the request backlog down to less than 24 hours, with just a few slip ups of a few days' wait while I was away.

For the past few weeks I’ve been teasing a new feature to the URL claim system - this will come soon! Below I’ll outline the timeline of near future plans.

Ongoing Things

External Images

Discord has officially ended support for remotely hosted images, due to rising costs. This means that any URLs with Discord images within them will stop showing these images very soon, if they haven’t already.

In the more distant future we may be able to consider direct image hosting. However this is both difficult and expensive at the scale Rentry demands and would require even more that we ensure we are protected from liability, if someone uploads something they shouldn’t.

PDF/PNG Exports

The export server spends nearly all of its time burning away all 6 CPU cores to produce images for thousands of requests each day. While most of the time it works fine, if the number of requests grows too high, it starts to fail. If you’ve had this issue, just try again shortly afterwards.

Fixing this is not easy - besides the simple (and expensive) option of upgrading the server. With so many remotely hosted images, saving caches has a big problem - when should it be refreshed for each URL? It’s possible that the image hosted remotely can change at any time and checking for this is another problem in itself.

I’ll continue to try to find better solutions for this.

Patreon / Ko-fi

We’ve been quiet on the plans for this ever since the last blog post and that is for a number of reasons:

Quite a number of you are keen on a Patreon, but do not have the means to support it.
We’re concerned that the monthly income from a Patreon would significantly drop after the first few months.
I really want to avoid the possibility of a failed Patreon, where we switch and then have no choice but to switch back due to financial difficulties.
The previously suggested plans had a bit of a fatal flaw: what if the secret code was released to the other community?
Currently Rentry is run as a service that is freely available, which means that it is perfectly acceptable that I simply do my best regarding site performance, updates, URL claims and responding to messages. If this is changed, I will need to fully commit to the highest level of service, in order to earn this continued support. While I think it’s currently at a high standard, and Sarah’s work raises it even higher, I’m concerned that a slight slip up might lead to letting supporters down.
We’ve not had a single complaint or even heard a mention about the ads since December.
It’s not clear how much of a difference it will make to Rentry’s income if ads were fully switched off for you guys. I’m worried it will cause a significant enough drop that we would get into trouble again.
Currently, we are making enough to pay for everything outlined in the previous blog post, plus Sarah’s salary, which puts us in a comfortable position to continue on forever.

I’d like to be open about what currently causes ads to show or not show, as perhaps a reasonable ruleset can start us down the right path. Currently, the rules for ads are as follows:

If a post has less than 5 views, do not show.
If a post was first created before the 1st of April 2023, do not show. (Making URL claims more valuable)
If the current user created or edited any posts within the last 24 hours, do not show.

For now, I’d like to float a new proposal that is aimed at solving all above issues:

Rentry creates both Patreon and Ko-fi. For every $5 donated, a person may set a URL as being ad-free for 1 year. This can be their own URL, or anyone else’s and setting the same URL multiple times will cause the effect to stack, rather than reset.

Additionally, supporters themselves do not see ads on any URL.

For example, if donating $5 a month for 1 year, a person can build up to keeping 12 URLs ad free, or to making a single URL ad free for 12 years.

Please email or Twitter DM your thoughts to this!
support@rentry.co
@rentry_co

Unacceptable Content

A number of you have recently reported a certain type of page to us, ‘callouts’ on other people. While we support the publication of truth, we have no way to verify statements made about other people, and if we don’t take action following a report, we potentially start to become liable for libel.

For the most extreme callouts - if the statements are true - perhaps the police would be a better place to turn?

Exciting Things to Come

With all these things going on, my mind has still been racing ahead to the new things that can be added in to Rentry. A number of you have sent in some suggestions which as always are very helpful, and go towards painting the picture of what Rentry should look like in 6 months to a year’s time.

Content Detection

Back to the present, for the last few weeks I’ve been focusing on a major improvement to the illegal content detection system. (Not the most exciting for you guys admittedly, but important for Rentry’s survival.) For years now I’ve been collecting ideas on an extremely high level and powerful scanning system and in March I decided to finally bring it into reality. This system goes beyond simple keyword matching and introduces multiple techniques that help to defeat intentional obscuring of information. For example, the following simple variations of ‘stolen credit card’ can often defeat detection systems:

s..t..o….l…e..n…c..r..e..d..i..t.c..ar.d
St013n cr3d1t c3rd
Sto–en cr—it -ard
𝓼𝓽𝓸𝓵𝓮𝓷 𝓬𝓻𝓮𝓭𝓲𝓽 𝓬𝓪𝓻𝓭
🆂🆃🅾🅻🅴🅽 🅲🆁🅴🅳🅸🆃 🅲🅰🆁🅳
𝓈ᵗ๏Ⓛ乇几 ¢г𝓔ｄ丨t ᶜ𝐀𝓻𝓓
ᔕTOᒪEᑎ ᑕᖇEᗪIT ᑕᗩᖇᗪ

The new system can detect all of these, and much more. This is all without turning yet to AI, which will be a future improvement and which will benefit hugely from the current ability to pre-process each post and translate it to plain text.
I was hoping to be completing this system this week, but have overrun a little. New target: next week!

Image Control

Currently in the testing phase, this will add a little more ability to control how your images look on your URLs.

Quite a few of you are losing your URLs because you share the edit code with someone that turns out to be untrustworthy. This new tool will help prevent this.

Bug Fixes

There aren’t many known bugs in Rentry, but the small ones that remain will be fixed shortly.

New Edit code loss prevention

The auto edit code vanishes a bit too quickly when making a new Rentry page. It’ll be made a little ‘stickier’ to help prevent this.

URL Claim Improvements

The URL claim system has a rule: the URL must be put to use after being claimed. Soon this will be enforced.

A major new feature for page customization

Plans are being finalized for a new way to take complete control over a page, without resorting to writing HTML and CSS.

A major extension to the major new feature for page customisation

Part 2 of the major feature is being prepared.

Closing

Thank you all for sticking with us, I hope you join me in looking forward to what the next few months has in store for Rentry.

- Matt