How a 64k intro is made (2017)

205 points by aragonite 1 year, 7 months ago on hackernews | 38 comments

[OP] aragonite | 1 year, 7 months ago

Past HN discussions:

- https://news.ycombinator.com/item?id=14392305 (2017 - 60 comments)

- https://news.ycombinator.com/item?id=16842576 (2018 - 60 comments)

Here is a another great place to get started if you want to get into 64k size coding: https://64k-scene.github.io/
I came here to say that :)

In particular, we have a list of making-ofs like this one: https://64k-scene.github.io/resources.html#making-ofs

It's probably good to know that the focus in the demoscene has shifted away from 64k more towards 4k/1k intros and 'regular' demos. The amount of work required to do a high quality 64k is so huge that, especially now that the demoscene is not as big as it used to be, a lot of people don't think it warrants the time investment anymore.

(Btw, the best way to prove me wrong is to write a 64k intro about it)

Actually a bunch of people that do still like them... it's just that we don't have the infinite time we had at our disposal in our teens that makes us not ever be finished :P
Isn't that the same issue really? The demoscene seems to be in decline because there's hardly any new blood coming in, the teens who would have time to commit to a big production today just aren't interested.

llmblockchain | 1 year, 7 months ago

Teens have been so accustomed to immediate gratification and short form content they don't possess the willpower to focus on something truly difficult. You can't watch a few shorts and bust out a 4k demo over the weekend the same way you can "make minecraft in 24 hours".
Maybe that's part of it, but I think not having any personal experience with the constraints of early computing is also a factor. The demoscene does try to keep that spirit going on modern platforms with artificial constraints like "Windows exe limited to 64kb" but I think it's harder for someone coming in fresh to appreciate the appeal of that handicap. They definitely won't appreciate the actual retro computer targets like the C64 and Amiga, why would they, those systems came out before they were born and unlike early game consoles they have almost no lasting appeal outside of nostalgia (sorry Amiga fans).
Teens addicted to only instant gratification were never the ones to produce really good stuff in the demoscene anyhow, they will put countless hours into other arenas.

Also back in the mid 1990s maybe the demoscene _was_ instant gratification to get started? You could start doing graphics with "mov eax,13h / int 10 / mov ax,a0000h / mov es,ax / mov di,0 / (loop with stosb).. ".

To get started today with graphics you either setup openGL/d3d with their weird API's with decades of legacy, use something third party (and learn linking/libraries) or write a 3000 LoC hello-vulkan (Or use Unity,etc but then up until 5 years ago you'd get shouted out of the scene due to elitistic attitudes).

I don't think that the time investment (per-se) is the biggest issue for the demoscene, 64k is the most time consuming category in many ways but also becoming a niche _again_ today.

It'd be easy to write an essay on it, but pirates games and cracktros not being as visible in the days of cheap games, copy/lan parties as focal points of computer usage in the 80s/90s, lan party consolidation and separation of demoscene from lan parties, _lack_ of funding for commercial graphics programming (games) before y2k in Scandinavia compared to available talent, inflexibility of early 3d accelerators were all factors in making the demoscene a thing and later it's decline.

My point exactly :)
It's also worth noting that the expectations around 64k got higher. We can do so much more in 64k than it was possible in the past. Compression tools have improved. If you take the famous fr-08 intro from Farbrausch and recompress it with modern packers, you would get back many kilobytes. So I think the time investment needed to make a top 64k has increased.

The source code of all my 64k intros has been shared (the last one was added two weeks ago): https://github.com/ctrl-alt-test/demo-archive/ and my plan is to share more code to help people get started. Creating demos using procedural generation is incredibly fun.

I've toyed with doing writeups of some of my demos, but one aspect I've struggled with is how the explanation of how it works is often severely departed from the process of creating it.

The writeup here appears to be fairly natural, and maybe this is the true process for something as large as 64k, but I mostly do 192 byte demos, the largest I've ever done is 512 and 1024 byte. I think there is a difference at this size where the size very much takes charge of the creative process - and that can be quite interesting because you end up doing a lot of experimentation and accidentation and discover things you wouldn't normally be pushed to and can pivot the entire demo. However when deconstructing the result in order to explain it, this juts gets lost and I've struggled to figure out how to convey this duality. To only post a "how it works" would omit the part where all magic happens, yet that's hard to explain on it's own, and hard to combine with the "how it works".

codetrotter | 1 year, 7 months ago

Would be cool to see a full video of the process from start to finish, if you’re up for it.

That way people can get a better impression of it.

I've actually tried that on Twitch, it was fun for another reason: viewer involvement. But as a format I found it didn't really work well because for my most impressive demos 90% of the time I'm just staring intensely at tiny fragments of code and trying to rearrange and run it in my brain - this is not interesting to watch believe me, I suppose one could condense the video to only show the ends of those segments and just talk out-loud about the concrete thought that has coalesced at the end of it and the actions that are now being taken.

codetrotter | 1 year, 7 months ago

> I suppose one could condense the video to only show the ends of those segments and just talk out-loud about the concrete thought that has coalesced at the end of it and the actions that are now being taken.

I like that idea.

This is the same for any creative process. Any painter could describe how they’re reasoning about a composition and what-not. But ultimately each individual needs to digest technique and make it their own.

Writeups like this are great though because it can at least share the tools used and give folks some place to start.

Maybe a "why it works" post would be useful, as in "this part of the demo does x because I was trying to do y but ran into this issue, which caused me to experiment with z and then a side effect inspired me to switch abc to def", etc. Personally I think explaining the creative process is more interesting than explaining the end result.
> Personally I think explaining the creative process is more interesting than explaining the end result.

Thanks, I think this is right, to focus on the process instead (and commenters indigochill and lawn) which I think are describing the same type of writeup.

Perhaps my error was starting with "how it works", whereas I probably should focus on the story of the process and add the "how it works" where it makes sense in the story. I also don't really try to record the process at the moment, it's just from memory.

I know nothing about the subject but an idea is to use a "let's make a demo" story to, as you say, capture the important journey instead of the end result.

It takes more effort than a simple "how it works" explanation but if done correctly I think it should explain how the demo works while also capturing the magic of the creation process.

How long does it take you to make a demo? Might be cool to do a live streaming, or just record the process even if not live.
Sometimes I do it in one sitting, sometimes that's quite long, like 12 hours or more! Other times I will revisit it over days or months - You'd be surprised how much thought and transformation can go into 192 characters. It depends how much potential room for improvement it "feels" like there is left, you get a sense for it, sometimes you just know it's done, others you're never quite sure when to stop trying to push harder.

I've commented in a sibling my experience live streaming which was interesting. Another Demoscener who has given this far more concerted effort on Twitch is KilledByAPixel, I think they are recorded somewhere maybe on youtube. My two attempts are lost to time.

As an example (yes shameless plug) I wrote this one recently over a couple of long evenings, I'd estimate 8 hours maybe:

https://www.dwitter.net/d/31805 (runs fastest in Firefox)

But there are some familiar micro-raymarching techniques I've already developed and reused in this. The bit that took most of the time in this one was figuring out how to fit binary search marching that was required to support large differences in marching distances and textures without producing excessive moire effects.

indigochill | 1 year, 7 months ago

Although a different context, I always find the writeups of cybersecurity CTFs that go through the "What I was thinking, what surprises I encountered, how I pivoted" process both more enjoyable and more enlightening than writeups that simply explain the solution as if it was known from the beginning.

At the same time there might be some editing since there might have been approaches tried that didn't go anywhere, and whether that's interesting/relevant to the reader is probably a judgement call from someone who knows the domain and whether those dead ends might have been natural things to try in that specific context.

> the explanation of how it works is often severely departed from the process of creating it

This description rings true for a whole swath of creative endeavors, including ones like demos that appear technical on the surface: optimizing an algorithm, conducting scientific research, building an engine, constructing a mathematical proof, and so on.

High-quality 64k intros usually need a film-like production process nowadays, except it is harder because you have to fit it into 64k later, it typically means procedurally generating everything as well as using the appropriate tools (runtime packers, compiler options, etc...). It is, I think, the most demanding demoscene category.

There is much less commitment in a tiny intro. You don't have days of preparation, storytelling, art direction, etc... to stick to. If, when optimizing your intro, you stumble on something better, you are better off throwing everything away and build on that instead, "everything" is just 192 bytes after all. I didn't know 192b was a thing BTW, I am more accustomed to powers of two, like 256b.

It's 194 bytes I keep getting it wrong :D yeah it's a strange size, the short version is SMS (140 bytes) => twitter => dwitter => unicode packing of ascii with JS decoder (which could not be undone because the limit was defined as unicode points) yields 194 bytes.

> If, when optimizing your intro, you stumble on something better, you are better off throwing everything away and build on that instead,

Exactly this, but because of the extreme size limitations you often end up forced to iterate out of frustration trying to fit something, and then something else weird but interesting will often pop out of that process that you wouldn't have directly thought of.

DrewADesign | 1 year, 7 months ago

I straddle the tech/art gap as a tech artist in 3D VFX & Animation. I've done work as visual as branding and identity, and as technical as network performance and a back-end web development. I'm also a technical writer for a commercial rendering engine, which is pretty different, but I've some experience reading and writing about these things.

Technical folks often approach process write-ups as they would an incident report, or a tutorial. Rarely a pleasant read! A little prep before you start yields major improvement: First, decide exactly what you're saying, and to whom. For a big one, having a "story" part, a prosaic technical write-up, and a reference section can fit multiple people's needs. Then, consider exactly what your audience wants and needs to know about your message: try not to be overly broad. Make that into a simple narrative and story arc, and expand it into an outline. It sounds like boilerplate, but it's more like scaffolding. Correctly applied, it steers you towards a concise, compelling piece without endless dickering and waffling about what to say next. Start with the broadest, least specialized audience-- adding details is easier than editing them out.

If you write your demo in a recording editor, then when you're done, you can go back and watch the playback of all your edits and add commentary.
Interesting idea, thanks!
I'm wondering if there are similar demos featuring graphics and sound with strong constraints but written in Python?
Not really, the only way it makes sense for a high level language to be used in a constrained competition is if the rules are specifically set up so that the runtime doesn't count towards the size limit. There are such competitions for Javascript, where you get 64kb of HTML to play with and the size of the browser itself is "free", or for Lua via the PICO-8 runtime, but I don't think anyone has done it for Python.
There's also https://dwitter.net which has a limit of 140 characters of javascript, and the site is fairly active.
Is it 64K period, or 64K plus access to gigabytes of external libraries?
64kB is the size of the binary file. You can rely on an operating system (Windows, Linux...) with its graphics drivers.

Basically, you can use the GPU and you can use most of the 64kB to generate procedural content.

It's an exe <= 65536 bytes. A modern OS does have gigabytes of external libraries, but the functions in them are more "support every graphics and sound chip out there" than "draw cool spaceship" and "play a funky beat". They're not allowed to obtain anything from the internet, nor anything else beside what comes as standard with the OS.

Everyone gets the same base OS and compete against _each other_, so nobody gets a special advantage. If someone finds a clever way to use the base OS for more bytes (such as reusing the standard Windows fonts or MIDI soundfont or CAB compressor), then so does everyone else, and they use the saved space for even more cool and impressive stuff next release.

Here's some tricks used for 4kb demos (some of which are applicable to 64kb demos too):

https://in4k.github.io/wiki/aulds-4kb-introduction

https://in4k.github.io/wiki/aulds-4k-synth

https://in4k.github.io/wiki/exe-packers-tweakers-and-linkers

The executable has to be 64k or less and has to run by itself on a clean install of your OS of choice.

There may be "gigabytes of external libraries" included in the OS, but these are necessary for using current-gen hardware. Libraries and assets that are not just for interfacing with the hardware and basic stuff like opening a window are rarely taken advantage of, except sometimes for text rendering. System decompressors used to be used, but nowadays, productions come with their own specialized unpackers (ex: kkrunchy) that are much more powerful yet smaller than anything that comes with the OS. EDIT: I see MIDI being mentioned, and it is indeed still used. But for 64k, there are softsynths that sound way better than system MIDI and only take a small part of the size budget, sometimes going as far as physically modelling instruments. 64k is crazy.

These 64k intros could be made run on the bare metal with not much difference in size if there was hardware for it. But the way modern hardware is designed, it is impossible. If you really insist on going "bare metal", try an Amiga ;)

Here I thought it was going to be a 64k Amiga or .COM DOS demo. This seems to be a very expensive and elaborate demo that is out of general reach.
Coming from the "old ages" when a PC didn't even feature a (3D capable) GPU and all was about getting all the possible juice of the CPU... shaders now allow incredible things in a very small size, as long as you stick to procedural content. shadertoy.com features an insane amount of great samples.