Thinking in an array language (2022)

96 points by tosh a day ago on hackernews | 20 comments

tomhow | 23 hours ago

Previously:

Thinking in an array language (2022) - https://news.ycombinator.com/item?id=38981639 - Jan 2024 (152 comments)

Thinking in an array language - https://news.ycombinator.com/item?id=31377262 - May 2022 (66 comments)

IshKebab | 21 hours ago

I still haven't used K/Q/etc. because they look insane, but the more I read about them the more it seems like they are basically the maths equivalent of regexes. Super terse and powerful. Pretty much write-only. Very useful for interactive use, definitely. But if you find yourself hitting "save" on a regex that's a red flag and if most of your program relies on regexes something has gone very wrong.

redrobein | 21 hours ago

They are far from regexes. I feel like you're equating something you don't understand to something else you find difficult. It's a language like any other. Infact it's far less verbose and shorter to read. If you've spent enough time with an arraylang you start to recognize "words" or sequences of glyphs as some complex function[1].

You might have a better time trying something like Uiua[2] or BQN.

[1] https://aplcart.info/

[2] https://www.uiua.org/tour

jimbokun | 16 hours ago

> It's a language like any other. Infact it's far less verbose and shorter to read. If you've spent enough time with an arraylang you start to recognize "words" or sequences of glyphs as some complex function[1].

Like regexes!

IshKebab | 4 hours ago

> to something else you find difficult

I didn't say I find regexes difficult. I don't.

casey2 | 20 hours ago

The bigger problem is that you often end up using more functions, worse algorithms or higher algorithmic complexity, creating your data structure at runtime, since the only datatype you have are arrays.

It's similar to the problem with unix/plan9, lisp or smalltalk. There is it turns out, too much of a good thing.

Such systems are great for learning, but I would make the mistake of building software on them

jimbokun | 16 hours ago

Common Lisp supports many data structures besides lists, and the same is true of many other Lisps as well.

And Unix is easily the most used family of operating systems on Earth. Every iOS and Mac OS device has a Unix at its core, same with Android, countless computers in data centers running a Linux distribution, probably many other categories of devices as well.

I would hesitate to say building software on Unix is a mistake.

[OP] tosh | 10 hours ago

you don't have to represent everything with arrays (k has arrays, dicts, tables, lists)

that said: finding array representations and transformations is usually a great idea for clarity and performance

many problem domains can collapse unnecessary abstractions down to the metal, but many languages and ecosystems make it difficult to "see" when that is the case

array languages have a way of nudging you into fewer abstractions and modeling things in dense arrays where applicable, and you will be surprised how often that is applicable

implementations can be highly efficient and will often make use of SIMD (and other optimizations)

in many cases an idiomatic, simple array language program will beat a naive solution written in C

BQN documentation has a great read on why

https://mlochbaum.github.io/BQN/implementation/versusc.html

the gist is writing in an array language helps because the interpreter has an easier time mapping high level array transformation concepts to optimized vectorized kernels than a C or C++ compiler looking at algol-like code

xelxebar | 16 hours ago

> Pretty much write-only

I've written and worked on real-world APL applications, and this doesn't fit my experience. APL allows writing some of the most readable code out there, specifically because it makes whole-application architecture imminently more visible.

I like APL because it keeps me from bike-shedding on boilerplate or ceremony and forces me to consistently think about fundamental issues in the problem domain.

That said, APL by beginners (including myself, years ago) can be pretty terrible and effectively write-only. The steep learning curve is very real, and learning to write good APL is synonymous with learning good application architecture and UX.

2pEXgD0fZ5cF | 20 hours ago

Giving APL a quick glance I find the topic at least intriguing.

So if I understand this right K is one of the more popular/recommended APL-derivatives nowadays. K is proprietary but there are a bunch of compatible implementations which are themselves open source. Is that correct?

Which APL derivatives (with an active/healthy userbase) besides k would you recommend to take a closer look at?

Jtsummers | 20 hours ago

https://www.jsoftware.com/ - J has a pretty active community, or at least used to.

loganmhb | 20 hours ago

I’m an array language novice but my favorite one so far is BQN. Much better documented than K derivatives. Not sure you could characterize any array language as having a “healthy userbase” though.

xelxebar | 16 hours ago

> So if I understand this right K is one of the more popular/recommended APL-derivatives nowadays.

Only APL and K see real industry use. Both are actively developed. K is really a family of mutually incompatible languages, called K3, K4, K5, etc. Financial applications and most of the development is on K4, as far as I understand.

> Which APL derivatives (with an active/healthy userbase) besides k would you recommend to take a closer look at?

But why not APL? Dyalog has a team actively working on their interpreter, and industry has mostly converged around Dyalog APL now. If you want something well-worn through real-world use, then it's basically the only choice, unless you have money to drop on K4 or Shakti. Also, the literature is dominated by APL.

If you're more interested in array languages for their mathematical appeal and for tinkering on puzzle problems, then BQN will probably suit your tastes. If you don't like the symbols and want something open-source, then J is a strong option. If you wonder what kind of baby APL and Forth would have, then Uiua is just for you.

Note that while J and BQN can be considered APL variants, K is somewhat of a separate beast.

Have fun!

cratermoon | 14 hours ago

[OP] tosh | 11 hours ago

re APL:

BQN is a new take on APL w/ high performance implementation

https://mlochbaum.github.io/BQN/

Dyalog APL is a well supported proprietary APL implementation

https://www.dyalog.com

re open source k implementations:

there are many by now

here is a current non-exhaustive list grouped by k dialect they orbit (to my understanding)

k9

ktye (Go w/ Go, wasm, C targets) https://github.com/ktye/i

k6

ngn/growler (C): https://codeberg.org/growler/k

oK (JavaScript): https://github.com/JohnEarnest/ok

kiwi (Zig): https://github.com/kiwi-array-lang/kiwi

k3

kona (C): https://github.com/kevinlawler/kona

gk (C): https://github.com/cmh25/gk

ksharp (C#): https://github.com/ERufian/ksharp

k-like

goal (Go): https://codeberg.org/anaseto/goal

klongpy (Python) https://github.com/briangu/klongpy

I'm working on kiwi.

kiwi has a focus on supporting Apple Silicon and can make use of Apple Silicon GPU (via MLX and Metal) as well as the Accelerate framework to speed up some workloads.

kiwi also comes with companion apps for iPhone, iPad, Mac and Apple watch in case anyone wants to run k on a watch:

https://kiwilang.com

PufPufPuf | 11 hours ago

Are there any open-source software projects written in an array language? I've only ever seen short code examples and heard of fables that "K powers the world's financial systems". I want to see a "real-world" codebase.

ofalkaed | 10 hours ago

dzaima | 6 hours ago

In BQN, I've made https://codeberg.org/dzaima/bqn-smt/ (SMT engine bindings, plus various utilities, and a RISC-V & x86 superoptimizer of varying amounts of completeness); ~4KLoC (+1KLoC of tests). Might not fit your "real-world" as I am mostly its only user (and many things are undocumented), but it does have a good amount of non-trivial things aren't necessarily particularly array-y.

Can also look through APL github repos: http://github.com/search?q=language%3Aapl&type=repositories

kcroarkin | 5 hours ago

I am writing a voxel game in Dyalog APL where I call a low level graphics API directly in APL and perform nearly all of the core logic in APL (Terrain generation, generating geometrical representations, collision, frustum culling). You can see it here: https://github.com/namgyaaal/avoxelgame and a demo of me playing it here: https://www.youtube.com/watch?v=LgNkSdAXS_c