M4 is awful and needs to be buried. Macros are useful as a concept, but M4's take has no redeeming qualities. Whatever DIY thing you invent in 5 minutes is likely to be more intuitive and less fragile.
Unix has so much crappy stuff that got in 40 years ago on a whim, and we just keep rolling with it because it's already there. The ossified base also makes getting updated tools and libraries cumbersome, perpetuating the problem.
Yeah, I was full-on agreeing with the article until the escaping mechanism, which pushed it over onto the "I think this is dumb" side again. Now it breaks if someone uses two tildes, which seems very brittle to me.
Guix is nice if you enjoy Scheme, I actually havethreeposts about it. But for my VPS, I was explicitly aiming for OpenBSD, because it's a system I was hoping to learn and I wanted something very stable and safe by default.
Solving this is non-trivial. While m4 does have something akin to escaping, it doesn't quite work like it does in other languages and it's brittle enough for me not to even bother with it in this post.
Instead, we'll reach for another trusty Unix tool, sed
I ran into a similar issue when my blog's custom static site generator was written in shell scripts+M4.
I wrote a post about how I wrote the generator and because I had included snippets of M4 in the post, it put itself into an infinite loop at one point.
I too added some sed as a bandaid but I ended up scrapping using M4 because it just became a can of worms in escaping certain stuff...
However, using M4 for simple configs here is a pretty fantastic idea I might steal.
I desperately wish Sed had an option to provide the pattern and the replacement as separate strings. I often end up writing sed "s#somepattern#$SOME_REPLACEMENT_STRING#" but obviously that only works if the replacement string doesn't contain hashes. It should also have a "fixed replacement string" mode where the replacement string doesn't have syntax for references to parts of the pattern.
(or sub(PATT,RAPL) for replacing only single instance per record (line))
Downside: You cannot refer to capturing groups in replacement string, & (matched pattern) is the only meta thing.
Upside: Ditto, thus smaller escape surface. Use \& and \\ for literal & and \. [^1]
It's worth to remember that awk uses ERE by default, which in sed requires -r option.
It does not provide convenient in-place file editing unless you're willing to go with GNU awk.
[^1]: But awk is actually finicky too, due to use of variable assignment, backslashes require additional escaping, which can easily become less readable, thus better way is to set variables via environment before running awk, because then you don't need additional layer of escaping.
I once got not too bad with M4. The stuff actually makes sense.
I wrote 10 to 15 lines for autoconf and it worked great. It was generating -W option tests for gcc from a list to avoid repetitions.
I lookef at the code a few months later and was avsolutely unable to even somewhat parse it. I've never felt so much like a stranger to my own code. I haven't even felt so much like a stranger to other people's perl code!
So yeah, you may feel fine using m4 and write readable code. But it's still m4 and you will regret it a few months later.
I'm pretty sure there are better languages or tools out there, but Perl is a strict superset of sed and awk, and it's included in the base OpenBSD install AFAIK[1]
[1] I know this because purists sometimes show up in the mailing lists demanding it be removed.
I went down this path myself and came to the same conclusion. I wanted to use it as a static site generator, and it seemed fun to use such an old standard part of UNIX to do it. I gave up on it, its quirks are not fun.
Instead, we'll reach for another trusty Unix tool, sed. If you've ever used s/old pattern/new pattern/ in Vim (or other editors and, for some inexplicable reason, Discord), you already know what sed is capable of. It takes a command and executes it on the lines of a file.
Surprisingly, the /s/pattern/replacement/ in-band editing command crops up in the most surprising places. My favorite instance was Skype, way back in the early 2000-teens.
I cannot give an authoritative answer to this. In my case, I want my VPS to be largely "self-sufficient". That is to say, I'd like to follow the OpenBSD philosophy and rely on tools already present on the machine rather than install extra stuff. This is both a fun challenge and also makes the system very "fire and forget."
I love this approach and try to follow it as well. Just using POSIX tools feels a bit spartan but the resulting longevity of your scripts (let's be frank -- it's mostly sysadmin-type scripts like the ones the article demonstrates) is well worth it.
Also, I understand OpenBSD makes extensive use of Perl, it being part of the base system and all, but I've never actually sat down to learn it. One of my colleagues in a previous job did sit down and learn enough perl to get work done on an extremely locked-down server (no python, ruby, etc.)
Once you're into generating configs and if you don't want to go full nix, there are also great systems that will keep your config data type-checked and well structured, but can generate any final format you want. For example https://nickel-lang.org/ can be serialised to httpd config if you want. And yes, you can still use perl or whatever if you prefer... but we should really just let m4 die already. It's not better than any other system.
kornel | 17 hours ago
M4 is awful and needs to be buried. Macros are useful as a concept, but M4's take has no redeeming qualities. Whatever DIY thing you invent in 5 minutes is likely to be more intuitive and less fragile.
Unix has so much crappy stuff that got in 40 years ago on a whim, and we just keep rolling with it because it's already there. The ossified base also makes getting updated tools and libraries cumbersome, perpetuating the problem.
jeeger | 17 hours ago
Yeah, I was full-on agreeing with the article until the escaping mechanism, which pushed it over onto the "I think this is dumb" side again. Now it breaks if someone uses two tildes, which seems very brittle to me.
b3n | 20 hours ago
This makes me grateful for GNU Guix, where I manipulate configuration with a full blown programming language.
rau | 17 hours ago
A full-blown programming language which also supports macros!
nemin | 17 hours ago
Guix is nice if you enjoy Scheme, I actually have three posts about it. But for my VPS, I was explicitly aiming for OpenBSD, because it's a system I was hoping to learn and I wanted something very stable and safe by default.
ThatsInteresting | 16 hours ago
I kind of admire them for living off the POSIX land but we have much better choices if not in the base install (perl) then in ports.
[OP] zk | a day ago
I ran into a similar issue when my blog's custom static site generator was written in shell scripts+M4.
I wrote a post about how I wrote the generator and because I had included snippets of M4 in the post, it put itself into an infinite loop at one point. I too added some
sedas a bandaid but I ended up scrapping using M4 because it just became a can of worms in escaping certain stuff...However, using M4 for simple configs here is a pretty fantastic idea I might steal.
mort | 20 hours ago
I desperately wish Sed had an option to provide the pattern and the replacement as separate strings. I often end up writing
sed "s#somepattern#$SOME_REPLACEMENT_STRING#"but obviously that only works if the replacement string doesn't contain hashes. It should also have a "fixed replacement string" mode where the replacement string doesn't have syntax for references to parts of the pattern.przemoc | 14 hours ago
awk can be sometimes considered in such cases, more verbose but may be seem less finicky (for some parts) than sed:
(or
sub(PATT,RAPL)for replacing only single instance per record (line))Downside: You cannot refer to capturing groups in replacement string,
&(matched pattern) is the only meta thing.Upside: Ditto, thus smaller escape surface. Use
\&and\\for literal&and\. [^1]It's worth to remember that awk uses ERE by default, which in sed requires
-roption.It does not provide convenient in-place file editing unless you're willing to go with GNU awk.
[^1]: But awk is actually finicky too, due to use of variable assignment, backslashes require additional escaping, which can easily become less readable, thus better way is to set variables via environment before running awk, because then you don't need additional layer of escaping.
Example:
which should return:
adrien | 14 hours ago
I once got not too bad with M4. The stuff actually makes sense.
I wrote 10 to 15 lines for autoconf and it worked great. It was generating -W option tests for gcc from a list to avoid repetitions.
I lookef at the code a few months later and was avsolutely unable to even somewhat parse it. I've never felt so much like a stranger to my own code. I haven't even felt so much like a stranger to other people's perl code!
So yeah, you may feel fine using m4 and write readable code. But it's still m4 and you will regret it a few months later.
gerikson | 17 hours ago
People will do anything to avoid using Perl.
nemin | 17 hours ago
I intend to learn it, but every time I skimmed it, it seemed intimidating with it's fancy operators, variable declarations, and argument juggling.
gerikson | 14 hours ago
I'm pretty sure there are better languages or tools out there, but Perl is a strict superset of sed and awk, and it's included in the base OpenBSD install AFAIK[1]
[1] I know this because purists sometimes show up in the mailing lists demanding it be removed.
dkl | 12 hours ago
"strict superset" means it will accept any valid sed or awk program and execute it, which I don't think is what you mean.
gerikson | 10 hours ago
You are correct, it’s a semantic superset:
https://www.cs.ait.ac.th/~on/O/oreilly/perl/learn/ch18_02.htm
dkl | 12 hours ago
I think it's fun to use m4 for a few days and see how it works. Experience will quickly acquaint you with the reasons why it is obscure.
Gracana | 59 minutes ago
I went down this path myself and came to the same conclusion. I wanted to use it as a static site generator, and it seemed fun to use such an old standard part of UNIX to do it. I gave up on it, its quirks are not fun.
pzel | 22 hours ago
Very fun read! About this bit:
Surprisingly, the
/s/pattern/replacement/in-band editing command crops up in the most surprising places. My favorite instance was Skype, way back in the early 2000-teens.I love this approach and try to follow it as well. Just using POSIX tools feels a bit spartan but the resulting longevity of your scripts (let's be frank -- it's mostly sysadmin-type scripts like the ones the article demonstrates) is well worth it.
Also, I understand OpenBSD makes extensive use of Perl, it being part of the base system and all, but I've never actually sat down to learn it. One of my colleagues in a previous job did sit down and learn enough perl to get work done on an extremely locked-down server (no python, ruby, etc.)
ThomasAdam | 13 hours ago
Heh.
Before I removed it, fvwm had the FvwmM4 module, for pre-processing fvwm config lines in M4:
https://man.openbsd.org/man1/FvwmM4.1
I know this was used by a few other core fvwm developers back in the day.
The only other use of M4 I've seen in the Real World (tm) was with Sendmail's configuration being in M4.
But who's still using that?
fanf | 7 hours ago
Don’t forget autoconf (or maybe try to forget it if you can)
viraptor | 15 hours ago
Once you're into generating configs and if you don't want to go full nix, there are also great systems that will keep your config data type-checked and well structured, but can generate any final format you want. For example https://nickel-lang.org/ can be serialised to httpd config if you want. And yes, you can still use perl or whatever if you prefer... but we should really just let m4 die already. It's not better than any other system.
nickgirardo | 2 hours ago
There appears to be an error in the first CPP example. I believe the final line should read
ABS(-5) // becomes -5 < 0 ? --5 : -5I think this is worth noting especially since the preprocessor wouldn’t make the simplification that
--5is5