Fancy Curl Sites

46 points by seb 16 hours ago on lobsters | 21 comments

JulianSildenLanglo | 12 hours ago

Finally, the last step is actually presenting the Curl version of your site to any user agent containing curl, which for me was just a few changes to my Nginx config, with other servers it'd presumably be equivalently simple. The main feature I wanted was an override header (X-Original: true) so you could still get an unmodified HTTP response like someone casually running Curl likely wanted, and I wanted everything to be served statically still.

Isn't this what the Accept-header is meant for? I.e. curl nytpu.com/gemlog/2026-03-18 -H "Accept: text/html".

nytpu | 7 hours ago

Yeah, that is true haha, I just wanted to add a specific header for fun

mcpherrinm | 4 hours ago

Yeah - curl and most other tools just do Accept: * or similar by default, so for the Let's Encrypt test sites, I serve an HTML version if you have text/html in your accept header like a browser does, but otherwise give you the plain-text version

ie, the two urls give you something different:

curl https://valid.x2.test-certs.letsencrypt.org/
curl -H 'Accept: text/html' https://valid.x2.test-certs.letsencrypt.org/

No ANSI/VT100 codes or anything, but it did give me an excuse to make some ASCII art :)

k749gtnc9l3w | 3 hours ago

And given there are non-browser non-curl fetchers, one can also look where text/x-ansi is relative to text/html

vbernat | 13 hours ago

Nice idea. The two other referenced sites mentioned in the post returns HTML when queried by Curl.

nytpu | 7 hours ago

:( they worked when I published the post, looks like they both must've reworked their sites and incidentally taken them down since then

tinthedev | 11 hours ago

I've skipped the show. curl tinthe.dev is HTML, don't see why to mess with the default. Seems like a super rare use-case. Are there people who curl sites?

Although, I did put in <link rel="alternate" type="text/markdown" href="/home.md" title="..."> as a hint for using Accept: text/markdown on the same URL. Or going to .md versions of pages. Did it a "sprint" of optimizing for other devices, screen readers, and LLMs. It is supported on the full page, posts included. Same point as the draft preview, less color.

Not sure if the X-Original header is the bit to use though. How many users will be comfy with cURL and headers, while not being comfortable using something like Accept: text/html (which is the defacto standard). Poking at the implementation, seems like you ignore Accept headers entirely.

Now I'm tempted to drop an invisible "instructions" block on top of curl, instructing for this alternate approach. But we're approaching diminishing returns there, hah.

DustyFuzzy | 11 hours ago

curl tinthe.dev is HTML, don't see why to mess with the default.

Fun, presumably.

tinthedev | 10 hours ago

Well, there was a tiny bit more in my comment, let me follow it up:

Are there people who curl sites?

I was wondering whether people do actually do do that for fun. First I've heard of it is the author and their inspirations. I'm genuinely curious.

nytpu | 7 hours ago

Well I make sure to mention it on my home page to let people know it's an option to try it, and I believe on the Fediverse/Mastodon Trinkey said they had their curl version quietly set up for months and then someone randomly stumbled across it and thought it was such a fun easter egg they contacted them about it.

I think people on computers (even tech nerd people) really just aren't curious and don't poke at things as much as they used to. Like, I'm friends with the owner of a long-running site which initially just seems to be a simple one-page gimmick site like crouton.net; but if you look at the (very short and easy-to-read) HTML and are at all curious you'll stumble across a subdirectory that's a huge (and unique) archive of old Flash videos. The owner said in the early 2010s she used to see lots of people (proportional to the actual visitor count) stumble across the hidden page; but nowadays few ever explore the page beyond what's explicitly presented to them.

tinthedev | 6 hours ago

Rereading my original post, I do sound a bit too negative - wasn't intended. I enjoyed poking around your implementation.

I guess I'm becoming a bit of a cynic... I generally do enjoy this kind of whimsy.

nytpu | 6 hours ago

Eh, your original point is valid, this isn't gonna be something that anyone would stumble across organically I would think.

But as for why in the first place, yeah, just for fun and whimsy. I also have nearly all of my site hosted on both Gemini and Gopher because it's just fun too, not because I'm expecting it to massively boost my readership and make me internet famous.

dlisboa | 7 hours ago

People jump out of airplanes for fun. We can be sure there are people who curl websites for Easter eggs.

iamnearlythere | 6 hours ago

Are there people who curl sites?

I've only memorized one, curl wttr.in gives a weather forecast in colored ASCII drawings. Unsure why it stuck with me..

A similar thing was showing a starwars movie in ASCII over telnet, probably a very long time ago.

jmiven | 5 hours ago

A similar thing was showing a starwars movie in ASCII over telnet, probably a very long time ago.

Yes, good old towel.blinkenlights.nl.

nc towel.blinkenlights.nl 23
# or, if you still have it and want to learn how to *quit* it:
telnet towel.blinkenlights.nl

JulianSildenLanglo | 10 hours ago

curl tinthe.dev just gives you 301 Moved Permanently, but slapping on a schema does resolve it.

tinthedev | 10 hours ago

curl defaults to http:// before the server even sees the request, so that 301 is the HTTPS push. I'd be curious if you've found a way around it.

jaculabilis | 8 hours ago

curl -L follows Location: headers.

JulianSildenLanglo | 10 hours ago

Not sure, and when I check my own website using curl it's actually even worse at it doesn't even give me a redirect output. I'll have to give it a think.

k749gtnc9l3w | 3 hours ago

Are there people who curl sites?

I curl sites by hand not that rarely, but generally to check at which stage things break for my curl → html-to-text pipeline. (I read a lot of stuff in Vim after having saved them to my drive)

If anURL marks itself text/plain via headers, then it gets to skip the html2txt part, of course.

jaculabilis | 5 hours ago

One hiccup with this is that, if someone really did use curl to browse the Internet, they probably have curl piped to something that expects HTML, e.g. readability --text | less. But if you serve text with ANSI to curl, then it won't parse as HTML and the page shows up as unreadable. Like LSP, the MxN problem of user agents and websites is usually solved by having one common format, HTML, and having each user agent adapt HTML to its idiom.