> An incoming HTTP request? it is a plain Clojure dictionary.
I learned to code in Python. Loved it. Dynamically typed dicts up the wazoo!
Then I learned why I prefer actual types. Because then when I read code, I don't have to read the code that populates the dicts to understand what fields exist.
Java doesn’t really have a nice interface for interacting with objects in general. Closure does have a nice interface for interacting with dictionaries. They have namespaces keyword symbols for keys which are much more ergonomic than typing strings, and they have lots of functions for modifying dictionaries. I think the big difference is in the philosophy of what the language thinks data is, and how the world ought to be modelled.
The two are not mutually exclusive. Clojure has namespaced keywords and specs[0] to cover that. (There is also the third-party malli, which takes a slightly different appproach.)
The advantage is that maps are extensible. So, you can have middleware that e.g. checks authentication and authorization, adds keys to the map, that later code can check it directly. Namespacing guarantees nobody stomps on anyone else's feet. Spec/malli and friends tell you what to expect at those keys. You can sort of do the same thing in some other programming languages, but generally you're missing one of 1) typechecking 2) namespacing 3) convenience.
Question: 1. Can a GET request have a non-empty request body?
2. Assuming you don’t know the answer to that question, will the type system you use be able to tell you the answer to that question?
This is a pretty simple constraint one might want (a constraint that only certain requests have a body) but already a lot of static type systems (e.g. the C type system) cannot express and check it. If you can express that constraint, is it still easy to have a single function to inspect headers on any request? What about changing that constraint in the type system when you reread the spec? Is it easy?
The point isn’t that type systems are pointless but that they are different and one should focus on what the type system can do for you, and at what cost.
Any statically-typed language with generics can express that by parameterising the request type with the body type. A bodiless request is then just Request[Nothing] (or Request[Unit] if your type system doesn't have a bottom type). Accessing the headers just requires an interface which all static languages should be able to express.
(1) note that “statically-typed language with generics” excludes a lot of statically typed languages, including C and Go (at least pre generics).
(2) this misses the meat of the question which is how to express that (eg) a GET request doesn’t come with a body and a POST request does. I suppose that you’re suggesting that one registers a url handler with a method type and that forces the handler to accept responses of a certain type. Or perhaps you are implicitly allowing for sun types (which aren’t a thing in many static type systems.)
(3) even in C++, isn’t this suggestion hard to work with. That is, isn’t it annoying to write a program which works for any request whether or not it has a body because the type of the body must be a template parameter that adds templates to the type of every method which is generic to it. But maybe that is ok or I just don’t understand C++.
F# has a feature called type providers that make this sort of bookkeeping between the database and the code less tedious, but even if you mess it up, static typing still gives you more safety than dynamic. If your code blew up because it should have accepted an identifier it didn’t, you know that the code has not been written to handle that case and can fix it. Alternatively, you can just choose to ignore this, and do what a dynamic language does. There is nothing stopping you from being dynamic in a static language, passing everything around as a map, etc.
Does “the request type has a body property” actually imply (1) though? In a language like C or C++ or Java, you could have a protocol like “body is always null on GET requests.” The question isn’t really about HTTP, that was just an easy-to-reach-for example, it is really about what having explicit types allows one to deduce about a program.
To be fair, an incoming request is, almost by definition, dynamic. It makes sense to have that as a map, since the main sensible thing to do on receipt is validation/inspection.
Granted, you may have a framework do a fair bit of that. Depends how much you want between receipt of the request and code you directly control.
Usually the approach in a statically-typed language is to transform your dynamic request into something that you know through parsing instead of validation. Here's a great article about this: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va....
That is a valid approach in any language. Static or not. Doesn't change my point that heavily. And it is all too possible to pick a bad parsing/binding language such that protocol changes in the request are now foot guns.
This is the second time I've seen the link above. And while I agree with the premise, the author clearly does not understand how to properly use the `Maybe` monad (a term that does not make an appearance!).
There is little use in wrapping a call in `Maybe` to then immediately unwrap the result on the next line. Doing so isn't really using the construct... One would expect the lines following the creation of `Maybe` to bind calls through the monad.
In the end I see almost no meaningful difference between their "Paying it forward" example and simply utilizing an `if` to check the result and throw. In essence the author is using a parse and validate approach!
Lexi absolutely understands how to properly use the Maybe monad. What you're saying to do here is the exact opposite of what this post is advocating for. You're talking about pushing the handling of the Maybe till later and the post is all about the advantages of handling it upfront and not having to worry about it anymore. You might want to read it one more time.
I understand. But what is purpose of `Maybe`? The reason one would reach to the above construct is precisely to offload (pushing to later) the handling of a value that may (or may not) be present at runtime such that a developer can write code assuming the value is always present and ignore the `Nothing` case.
Sure you can unwrap it right away, but that isn't necessary because you could also just "bind" the next function call to the monad (which is more idiomatic to the construct). You never have to worry about that value in this case because... well... that's the benefit of using `Maybe`.
I'm not super familiar with Haskell, but my sense is that the author is trying more to please the compiler (at a specific point in the program!) than simplify the logic. That is, they want a concrete value (`configDirs`) to exist in the body of `main` more than they want the cleanest representation of the problem in code.
In this case, it's to provide a better error message in case there's an empty list than `fromList` would provide.
> You never have to worry about that value in this case because... well... that's the benefit of using `Maybe`.
But you do, your entire program doesn't live in `Maybe` so at some point you have to check whether it's `Just a` or `Nothing`. Once again, the whole point of the post is to argue that getting out of the `Maybe` as close to parsing time as possible is preferable so you have a more specific type to work with after that. You also see right away what didn't parse instead of just knowing that something didn't parse, which is what would happen if you stayed in the `Maybe` monad for all your parsing.
Well... if your entire program is dependent on some input that may or may not exist at runtime... then it kind of does live in `Maybe`.
I have no issue with unwrapping a `Maybe` to throw an exception. But I do find it a bit ironic that the post is about parsing instead of validating, that the perfect construct is right there to exemplify how it could be done, but the author then chooses to eschew it and instead show examples of how validation could look.
The body of `main`, for example, could be refactored to something like:
maybeInitialized <- (getConfigurationDirectories >>= head >> initializeCache)
Which actually shows how `Maybe` can be used to simplify the system. If you want to unwrap the maybe at this point to throw, go for it! But the above is a much cleaner representation of the program than what author is trying to do (it's crystal clear how the cache might get initialized). I would expect "Parse don't validate" to be about how useful `Maybe` is to combine parsing logic into a functional flow vs. how validation leads to an ugly procedural approach.
I think you're referring to this part of the `getConfigurationDirectories` action, which has type `IO (NonEmpty FilePath)`:
case nonEmpty configDirsList of
Just nonEmptyConfigDirsList -> pure nonEmptyConfigDirsList
Nothing -> throwIO $ userError "CONFIG_DIRS cannot be empty"
The "meaningful difference" you're looking for is the type of `getConfigurationDirectories`. The previous version had type `IO [FilePath] `, which _doesn't_ guarantee any configuration directories at all. It did indeed check the results and throw. But it doesn't guarantee that all the `[FilePath]` values in the program have been checked. There are neither tests nor proofs in this code. In contrast, with the revised version, you can be certain anywhere you see a `NonEmpty FilePath` it is indeed non-empty.
The code I've quoted that checks which case we have, is the only place that needs to handle that `Maybe`. Or maybe `main`, if we want to be more graceful. The author (I wouldn't say I know her but I know that much) does know how to chain maybes with bind but it's not necessary in this example code.
My point is that if you are not chaining `Maybe` then the utility of employing the construct is unobserved. The entire purpose of using `Maybe` is to relieve the client from the need to make checks at every call for a value that may (or may not) exist. If you intend to immediately "break out" of the monad and (even more specifically) throw an error, you might as well just use an `if`.
I'm sure `main` could be written to "bind"/"map" `getConfigurationDirectories` with `nonEmpty`, `head`, and `initializeCache` in a way that puts the `throw` at the top-level (of course the above implementations may need to change as well). Unfortunately I'm not familiar enough with Haskell to illustrate it myself.
The purpose of Maybe is to explicitly represent the possible non-existence of a value which in Haskell is the only option since there's no null value which inhabits every type. The existence of the monad instance is convenient but it's not fundamental. The type of getConfigurationDirectories could be changed to MaybeT IO (NonEmpty FilePath) to avoid the match but I don't think it would make such a small example clearer.
There are numerous ways to redesign the function signatures, but I would imagine the simplest would be (again, idk Haskell syntax):
getConfigurationDirectories: unit -> Maybe [FilePath]
nonEmpty: [a] -> Maybe [a]
head: [a] -> Maybe a
initializeCache: FilePath -> unit
Notice `nonEmpty` isn't really necessary because `head` could to the work. The above could be chained into a single, cohesive stack of calls where the result of each is piped through the appropriate `Maybe` method into the next call in a point-free style. I cannot imagine how this wouldn't be clearer. e.g:
maybeInitialized <- (getCofigurationDirectories >>= head >> initializeCache)
That's the whole thing. Crystal clear. The big takeaway of "Parse don't validate" should be about the predominant use of the `Maybe` monad as a construct to make "parsing" as ergonomic as possible! Each function that returns `Maybe` can be understood as a "parser" that, of course, can be elegantly combined to achieve your result.
My critique is exactly that unwrapping the `Maybe` immediately in order to throw an exception is kind of the worst of both worlds. I mentioned this in a sibling comment, but my sense is that the author is more concerned with have a concrete value (`configDirs`) available in the scope of `main` than best-representing the solution to the problem in code. It is a shame because I agree with the thesis.
On the contrary the The NonEmpty type is fundamental to the approach in that example since it contains in the type the property being checked dynamically (that the list is non-empty). The nonEmpty function is a simple example of the 'parse don't validate' approach since it goes from a broader to a more restricted type, along with the possibility of failure if the constraint was not satisfied. The restriction on the NonEmpty type is what allows NonEmpty.head to return an a instead of a (Maybe a) and thus avoid the redundant check in the second example. The nonEmpty in your alternative implementation is only validating not parsing since after checking the input list is non-empty, it immediately discards the information in the return type. This forces the user to deal with a Nothing result from head that can never happen. Attempting to clean the code up by propagating Nothing values using bind is just hiding the problem that the validating approach avoids entirely.
You are misunderstanding the system. You can organize the logic into whatever containers you want, but the essence of the system cannot be changed.
You are already handling a `Maybe` type because it's possible for your input to not exist. Because the first implementation of `head` also returns a `Maybe`, it is possible to "bind" them together (I'm leaving out `IO` because I am both unsure of the syntax[0] and it is immaterial to the example):
head :: [a] -> Maybe a
head (x:_) = Just x
head [] = Nothing
getConfDirs :: Maybe [FilePath]
initializeCache :: FilePath -> Cache
useCache:: Cache -> Value
main :: ()
main = do
// you don't need concrete values here
maybeCache <- (getCofDirs >>= head >> initializeCache) // Maybe Cache
// one option
case maybeCache of
Just c -> useCache c
Nothing -> error "CONFIG_DIRS cannot be empty"
// another option
maybeValue <- (maybeCache >> useCache) // Maybe Value
[0] I have never written Haskell, so the above is my best-guess at the syntax given the snippets available (and no extra research)
The two functions `head` and `getConfDirs` are "parsers" because they both return `Maybe`. Contrary to
> Returning Maybe is undoubtably convenient when we’re implementing head. However, it becomes significantly less convenient when we want to actually use it!
It is trivial to use a reference to `Maybe` because it is a monad that it is specifically designed to be used more conveniently than the alternative approaches in the case when a value may (or may not) exist.
You might try re-reading it with some charity - the example's purpose isn't to teach the `Maybe` monad, but to remove the redundant check. To go into what `bind` does would be a diversion from the main topic (parsing vs validating).
But `Maybe` is specifically designed to remove redundant checks for a value that may (or may not) be present! That's the whole point of the monad! It seems rather unfortunate this isn't highlighted (or at least illustrated) doesn't it?
This is one of those self-inflicted Clojure problems. In Common Lisp you might use an alist or a plist for small things, but you'd definitely reach for CLOS classes for things that had relationships to other things and things that had greater complexity.
IIRC, the preference for complecting things via maps, and then beating back the hordes of problems with that via clojure.spec.alpha (alpha2?) is a Hickey preference. I don't recall exactly why.
No source to back this up, but my guess is that Clojure was driven by the need to interopt with Java so is to not get kicked out of production. This meant absorbing the Java object model. Shipping a language with both Java objects and CLOS and making them both play nice together sounds like a nightmare.
This comment helpfully explains many of the reasons Rich had for choosing immutable, persistent, generic data structures as the core information model in clojure (instead of concrete objects / classes): https://news.ycombinator.com/item?id=28041219
Not wanting to misquote the above / Rich himself I would TLDR it to:
- flexibility of data manipulation
- resilience in the face of a changing outside world
- ease of handling partial data or a changing subset of data as it flows through your program
Please note that no one (I hope) is saying that the above things are impossible or even necessarily difficult with static typing / OOP. However myself and other clojurists at least find the tradeoff of dynamic typing + generic maps in clojure to be a net positive especially when doing information heavy programming (e.g. most business applications)
Namedtuples FTW! A de-facto immutable dict with the keys listed right there in the definition to obviate all the usage head-scratching. Then, if you need more functionality (eg factory functions to fill in sensible defaults), you can just subclass it.
TBH I've never understood the attraction of the untyped dict beyond simple one-off hackups (and even there namedtuples are preferable), because like you say you typically have no idea what's supposed to be in there.
> Pure functions make code design easier: In fact, there’s very little design to be done when your codebase consists mostly of pure functions.
Ummm... I am a little bit fearful about your codebase.
If you don't see the need for designing your FP system it probably mostly means it is being designed ad hoc rather than explicitly.
If you are trying to compare to OOP system done right, you will notice that this includes a lot of work in identifying domain model of your problem, discovering names for various things your application operates on, and so on. Just because you elect to not do all of this doesn't mean the problem vanishes, it most likely is just shifted to some form of technical debt.
> Clojure is a dynamic language which has its advantages but not once I stumbled upon a function that received a dictionary argument and I found myself spending a lot of time to find out what keys it holds.
Dynamic typing is a tradeoff which you have to be very keenly aware of if you want to design a non-trivial system in a dynamically typed language.
It is not a problem with Clojure, it is just a property of all dynamically-typed languages.
One thing I don't like about all articles on clojure is that basically all of them say: ah, it's just like lisp with lists `(an (example of) (a list))` with vectors `[1 2 3]` thrown in. So easy!
But then you get to Clojure proper, and you run into additional syntax that either convention or functions/macros that look like additional syntax.
Ok, granted, -> and ->> are easy to reason about (though they look like additional syntax).
But then there's entirely ungooglable ^ that I see in code from time to time. Or the convention (?) that call methods on Java code (?) with a `.-`
Or atoms defined with @ and dereferenced with *
Or the { :key value } structure
There's way more syntax (or things that can be perceived as syntax, especially to beginners) in Clojure than the articles pretend there is.
(defn ^:export db_with [db entities]
(d/db-with db (entities->clj entities)))
(defn entity-db
"Returns a db that entity was created from."
[^Entity entity]
{:pre [(de/entity? entity)]}
(.-db entity))
(defn ^:after-load ^:export refresh []
(let [mount (js/document.querySelector ".mount")
comp (if (editor.debug/debug?)
(editor.debug/ui editor)
(do
(when (nil? @*post)
(reset! *post (-> (.getAttribute mount "data") (edn/read-string))))
(editor *post)))]
(rum/mount comp mount)))
Single engineers will pick clojure at companies , build a project in it, later that engineer will move on, now nobody can maintain this code so it’s rewritten in some normal language. I’ve seen that happen a few times. That code is hard to read and understand. This is why clojure will remain niche.
It could have been Go and Java programmer trying to understand it. Or it could have been some clumsy tool written in node which Go programmer finds hard to read and understand. Clojure's main advantage is that you can you can learn it very very quickly up to the point when you understand most of the code, the language is very very small compared to "five main languages".
> Single engineers will pick clojure at companies , build a project in it, later that engineer will move on, now nobody can maintain this code so it’s rewritten in some normal language
"Normal language"?
You mean, whatever language is most popular at the company. What's "normal" at one would be completely alien at another. Even things like Java. If you don't have anything in the Java ecosystem, the oddball Java app will be alien and will likely get rewritten into something else.
The reason Clojure remains niche is that some people somehow think it's not a "normal" language, for whatever reason.
You've seen a case where someone wrote something in Python that later devs could not understand and then rewrote it in . . . what? And you've seen that with Java?
There's a big difference between a developer going off and writing something in one of the top five most used languages in the world and doing so in Scala.
Yes. I've seen and contributed to dumpster fires in all of those languages. I would love to say it was all some rogue developer that crapped on things, but it is often just new developers. The more, the more damage.
1. picking a language/tool that a company doesn't have personnel with experience using it
2. picking a language/tool that is esoteric, which generally implies #1 as well.
#1 on its own isn't great, but generally when sticking in the java/python/ruby/javascript/php/etc...mainstream languages, there's a lot more documentation, and there's a higher chance that _someone_ in the company will have some familiarity. If nothing else, it'd be easier to hire a replacement for.
A higher chance, yes, but it doesn't matter much; what is tricky with most applications is the domain. Certainly, it's faster to go learn a language than to learn a new domain. To that end, you can get the whole team trained faster in a language than you can hire someone with experience and train them to the domain.
> Certainly, it's faster to go learn a language than to learn a new domain.
It's not only the language but the framework. For example I know javascript well enough but I now am quite a noob with Ember in my new role.
I would say the framework is just as important as the language, at least when doing web development.
You're kind of reinforcing the point though -- now you've got a whole team distracted by picking up a new language....why? how is it a good use of anyone's time? And it'll be a perennial training issue in the case of an esoteric language, because those team members will eventually turn over as well, meaning that you don't get to avoid either hiring or training a new person on it.
If it's just one component, implemented by a single dev, it really can make more sense to understand what it does and rewrite it in a language that's common in the company.
I'm not advocating NOT rewriting it. I'm just saying, back to the great grandparent's point, that the issue is a dev went rogue, NOT the language the rogue dev chose. The difficulty is the same regardless of the language the rogue dev chose; it's not that they picked Clojure, it's that they picked a language there was no organizational adoption of.
is it really hard to read (could be) or is it just that the average coder never saw lisp or sml and doesn't want to bother bearing the responsibility to learn something alien on duty ?
You need a team that wants to use Clojure. I wrote Clojure professionally for 2 years, and everyone at the company was excited about it and sold on the language. Even after 3-5 years of programming in it. Now, at a different place, we write in a different language, and even though I still love Clojure, I'm not gonna write some project in it, even if Clojure might suit it so well, because I know these people are sold on different language, and I'm not going to preach and I'm not going to make their lives more difficult by having to maintain some obscure codebase.
Minor point of order about the atoms: they're not defined with @ nor derefd with . If you're referring to earmuffs* that's convention not syntax (specifically for dynamically scoped variables, which could be atoms or anything else), and @ is indeed deref. (More specifically @x is a reader macro ish that expands to literally `(deref x)`.)
Agreed. These days I'm really fascinated by clojure and trying to learn clojure. Other than the project setup and repl and the editor (which I had considered), these weird characters are throwing me off.
What clojure really needs is some kind of opinionated framework or starter template, something like create-react-app. That has all these things figured out so a beginner like me can start playing with actual clojure, which documents all the steps to setup the repl and editor and what not. The last time I asked for this I was told about lein templates, they help but there's no documentation to go with those.
There needs to be some push from the top level. create-react-app was produced by facebook. Elm reactor (which lets you just create a .elm file and play with elm) was created by Evan the language creator himself.
tldr: There's a huge barrier to start playing with clojure that needs to come down and the push needs happen from the top level.
Yes, of course and I've got the book as well. The problem with the book is I got stuck on the very first code example in the book. I know there's a forum for the book where (hopefully) I can get my query answered.
My point is: these are all individual attempts (the book i mean) and there will always be something on page xyz broken and it can't be solved by individuals. To solve these problems, there needs to be constant time and money investment from someone serious (like facebook in case of create-elm-app).
Yes I agree there is a problem of a lack of institutional funding in the Clojure world. Luminus is a great tool but it is a bit sad that it is arguably the most production-ready web toolkit in the ecosystem and it is mostly the work of a single person.
There is some community effort to better fund the core infrastructure in Clojure through https://www.clojuriststogether.org/, hopefully they can continue to attract more funding developers and companies.
In general a lot of these issues could be alleviated if the community was just in general larger with more contributors. I think the Clojure community is quite welcoming to newbies in the sense that people are quite responsive, kind and helpful around the internet, in Clojurians Slack (try asking there btw, if you haven't yet and are still stuck at the start of the book), etc. But in other ways people seem averse to criticism or suggestions from outsiders. I think the Clojure world needs to do a bit of self reflection to understand why adoption is so low right now and honestly consider what needs to change to attract more developers and contributors.
You missed it, it has been there forever. But it says good and bad things about Clojure that its reference documentation is one of its weakest points.
The Guide/Reference split obscures a lot of information (do I want guidance on Deps & CLI or do I want reference on Deps & CLI?) and the guides where that gem is hidden randomly mix advanced topics (eg, how to set up generative testing), beginner topics (how to read Clojure code) and library author topics (eg, Reader Conditionals).
When you think about it, there is nearly no trigger to look at the guides when the information you need is there. Clojure is a weird mix of both well documented and terribly documented. All the facts are on the website, very few of them are accessible when required. The people who make it past that gauntlet are rewarded by getting to use Clojure.
is calling the method `.getAttribute` on the `mount` object – since it's a Lisp, it's in prefix notation. It also highlights how methods are not special and just functions that receive the object as first argument.
Finally,
@*post
is the same as
(deref *post)
and the `*` means nothing to the language – any character is valid on symbol names, the author just chose an asterisk.
Most of what you believe to be syntax are convenience "reader macros" (https://clojure.org/reference/reader), and you can extend with your own. You can write the same code without any of it, but then you'll have more "redundant" parenthesis.
Great article, love Clojure. Was trying to figure out what Nanit does. Might want to consider putting a link to the Nanit homepage on your engineering page. When just typed in nanit.com and saw the baby monitor tech, I thought maybe I went to the wrong place, until I saw the logos matched. Anyway, good read, but please put a link to your home page on your engineering site, or, put a 1 liner in the opening of your blog giving context to what your company does.
Being able to keep track of what data was where is the initial bump I had as well when learning Clojure. Unlike the author, I personally got used to it, and generally don't struggle with it anymore, but part of that is learning good habits on your code base where you make judicious use of names, doc-string, destructuring and have a well defined data model using records or Spec or Schema, etc.
The other one is just getting good at the REPL and inspecting the implementation for functions to quickly see what keys and all they make use of.
Something the article didn't really cover either is that it's not really the lack of static type checking that's the real culprit, its the data-oriented style of programming that is. If you modeled your data with generic data-structures even in Haskell, Java, C# or any other statically typed language, you'd have the same issue.
If Clojure used abstract data-types (ADTs) like is often the case in statically typed languages, things would already be simpler.
(defrecord Name [first middle last])
(defn greet
[{:keys [first middle last]
:as name}]
(assert (instance? Name name))
(println "Hello" first middle last))
(greet (->Name "John" "Bobby" "Doe"))
This is how other languages work, all "entities" are created as ADTs, it has pros/cons off course, which is why Clojure tend to favour the data-oriented approach where you'd just do:
Great article, love Clojure, unfortunately couldn't find any work with it when I tried, I managed to flop in the only interview I got :(
Still, I miss it sometimes when I'm writing C#.
I have had the pleasure of contributing to their code since we used their product at a previous company I worked at, and I must say I am sold on Clojure. Definitely a great language to have in your toolbox.
I found one of the perceived weaknesses of Clojure (in this article), it being dynamically typed, is a tradeoff rather than a pure negative. But it applies that tradeoff differently than dynamic languages I know otherwise and that difference is qualitative: It enables a truly interactive way of development that keeps your mind in the code, while it is running. This is why people get addicted to Lisp, Smalltalk and similar languages.
> To understand a program you must become both the machine and the program.
- Epigrams in Programming, Alan Perlis
Two of the big advantages of (gradually-) typed languages are communication (documentation) and robustness. These can be gained back with clojure spec and other fantastic libraries like schema and malli. What you get here goes way beyond what a strict, static type systems gets you, such as arbitrary predicate validation, freely composable schemas, automated instrumentation and property testing. You simply do not have that in a static world. These are old ideas and I think one of the most notable ones would be Eiffel with it's Design by Contract method, where you communicate pre-/post-conditions and invariants clearly. It speaks to the power of Clojure (and Lisp in general) that those are just libraries, not external tools or compiler extensions.
In 2021, I find it hard to justify using a dynamically typed language for any project that exceeds a few hundreds of lines. It's not a trade off, it's a net loss.
The current crop of statically typed languages (from the oldest ones, e.g. C#, to the more recent ones, e.g. Kotlin and Rust) is basically doing everything that dynamically typed languages used to have a monopoly on, but on top of that, they offer performance, automatic refactorings (pretty much impossible to achieve on dynamically typed languages without human supervision), fantastic IDE's and debuggability, stellar package management (still a nightmare in dynamic land), etc...
Can you elaborate why? To be honest, I don't have experience with large-scale Clojure codebases, but I have my fair share working on fairly hefty Python and Perl projects, and I tend to think that the parent commenter is mostly right. What makes you think they are incorrect?
Not who you are responding to, but the common idea that static types are all win and no cost has become very popular these days, but isn't true, it's just that the benefits of static typing are immediately apparent and obvious, but their costs are more diffuse and less obvious. I thought this was a pretty good write up on the subject that gets at a few of the benefits https://lispcast.com/clojure-and-types/
Just to name some of the costs of static types briefly:
* they are very blunt -- they will forbid many perfectly valid programs just on the basis that you haven't fit your program into the type system's view of how to encode invariants. So in a static typing language you are always to greater or lesser extent modifying your code away from how you could have naturally expressed the functionality towards helping the compiler understand it.
* Sometimes this is not such a big change from how you'd otherwise write, but other times the challenge of writing some code could be virtually completely in the problem of how to express your invariants within the type system, and it becomes an obsession/game. I've seen this run rampant in the Scala world where the complexity of code reaches the level of satire.
* Everything you encode via static types is something that you would actually have to change your code to allow it to change. Maybe this seems obvious, but it has big implications against how coupled and fragile your code is. Consider in Scala you're parsing a document into a static type like.
case class Record(
id: Long,
name: String,
createTs: Instant,
tags: Tags,
}
case class Tags(
maker: Option[String],
category: Option[Category],
source: Option[Source],
)
//...
In this example, what happens if there are new fields on Records or Tags? Our program can't "pass through" this data from one end to an other without knowing about it and updating the code to reflect these changes. What if there's a new Tag added? That's a refactor+redeploy. What if the Category tag adds a new field? refactor+redeply. In a language as open and flexible as Clojure, this information can pass through your application without issue. Clojure programs are able to be less fragile and coupled because of this.
* Using dynamic maps to represent data allows you to program generically and allows for better code reuse, again in a less coupled way than you would be able to easily achieve in static types. Consider for instance how you would do something like `(select-keys record [:id :create-ts])` in Scala. You'd have to hand-code that implementation for every kind of object you want to use it on. What about something like updating all updatable fields of an object? Again you'll have to hardcode that for all objects in scala like
case class UpdatableRecordFields(name: Option[String], tags: Option[Tags])
def update(r: Record, updatableFields: UpdatableRecordFields) = {
var result = r
updatableFields.name.foreach(r = r.copy(name = _))
updatableFields.tags.foreach(r = r.copy(tags = _))
result
}
all this is specific code and not reusable! In clojure, you can solve this for once and for all!
Your third point about having to encode everything isn’t quite true. Your example is just brittle in that it doesn’t allow additional values to show up causing it to break when they do. That’s not a feature of static type systems but how you wrote the code.
This blog post[1] has a good explanation about it, if you can forgive the occasional snarkyness that the author employs.
In a dynamic system you’re still encoding the type of the data, just less explicitly than you would in a static system and without all the aid the compiler would give you to make sure you do it right.
I think many peoples' experience is that most real world data models aren't as perfect as making up toy examples in blog posts. Requirements and individuals change over time. You can make an argument that in a perfect world with infinite time and money that static typing may be better because you can always model things precisely, but whether you can do that practically over longer periods of time should be a debatable question.
I've seen this article and I applaud it for addressing the issue thoroughly but I still am not convinced that static typing as we know it is as flexible and generic as dynamic typing. Let's go at this from an other angle, with a thought experiment. I hope you won't find it sarcastic or patronizing, just trying to draw an analogy here.
So, in statically typed languages, it is not idiomatic to pass around heterogeneous dynamic maps, at least in application code, like it is in Ruby/Clojure/etc. But one analogy we can draw which could drive some intuition for static typing enthusiasts is to forget about objects and consider lists. It is perfectly familiar to Scala/Java/C# programmers to pass around Lists, even though they're highly dynamic. So now think about what programming would be like if we didn't have dynamic lists, and instead whenever you wanted to build a collection, you had to go through the same rigamarole that you have to when defining a new User/Record/Tags object.
So instead of being able to use fully general `List` objects, when you want to create a list, that will be its own custom type. So instead of
val list = List(1,2,3,4)
you'll have to do:
case class List4(_0: Int, _1: Int, _2: Int, _3: Int)
val list = List4(1,2,3,4)
This represents what we're trying to do much more accurately and type-safely than with dynamic Lists, but what is the cost? We can't append to the list, we can't `.map(...)` the list, we can't take the sum of the list. Well, actually we can!
So what's the problem? I've shown that the statically defined list is can handle the cases that I initially thought were missing. In fact, for any such operation you are missing from the dynamic list implementation, I can come up with a static version which will be much more type safe and more explicit on what it expects and what it returns.
I think it's obvious what is missing, it's that all this code is way too specific, you can't reuse any code from List4 in List5, and just a whole host of other problems. Well, this is pretty much exactly the same kinds of problems that you run into with static typing when you're applying it to domain objects like User/Record/Car. It's just that we're very used to these limitations, so it never really occurs to us what kind of cost we're paying for the guarantees we're getting.
That's not to say dynamic typing is right and static typing is wrong, but I do think that there really are significant costs to static typing and people don't think about it.
I’m not sure I follow your analogy. I think the dynamism of a list is separate from the type system. I can say I have a list of integers but that doesn’t limit its size.
I can think of instances where that might be useful and I think there’s even work being done in that direction in things like Idris that I really know very little about.
There are trade offs in everything. I’m definitely a fan of dynamic type systems especially things like Lisp and Smalltalk where I can interact with the running system as I go, and not having to specify types up front helps with that. Type inference will get you close to that in a more static system, but it can only do so much.
The value I see in static type systems comes from being able to rely on the tooling to help me reason about what I’m trying to build, especially as it gets larger. I think of this as being something like what Doug Englebert was pointing at when he talked about augmented intelligence.
I use Python at work and while there are tools that can do some pretty decent static analysis of it, I find myself longing for something like Rust more and more.
Another example I would point to beyond the blog post I previously mentioned is Rust’s serde library. It totally allows you to round trip data while only specifiying the parts you care about. I don’t think static type systems are as static as most like to think. It’s more about knowns and unknowns and being explicit about them.
It is absolutely possible to have the same type for values that have the same shape.
You can have a `Map k v` that is a record that dynamic languages have that they call object/map.(make k/v Object or Dynamic if you want)
You don't need to create a new type with precise information if you just want that(no you don't need to instantiate type params everywhere). There is definitely limitations in type-systems (requiring advanced acrobatics) but most programs don't run into them and HM type system (https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_sy...) has stood the test of time.
> In a language as open and flexible as Clojure, this information can pass through your application without issue. Clojure programs are able to be less fragile and coupled because of this.
Or this can wreak havoc :) Nothing stops you from writing Map<Object, Object> or Map[Any, Any], right?
That's true! But now we'll get into what is possible vs what is idiomatic, common, and supported by the language/stdlib/tooling/libraries/community. If I remember correctly, Rich Hickey did actually do some development for the US census, programming sort of in a Clojure way but in C#, before creating Clojure. But it just looked so alien and was so high-friction that he ended up just creating Clojure. As the article I linked to points out, "at some point, you're just re-implementing Clojure". That being said, it's definitely possible, I just have almost never seen anyone program like that in Java/Scala.
Let me address your criticism from Scala's point of view
> they are very blunt
I'm more blunt than the complier usually. I really want 'clever' programs to be rejected. In rare situations when I'm sure I know something the complier doesn't, there are escape hatches like type casting or @ignoreVariace annotation.
> the problem of how to express your invariants within the type system
The decision of where to stop to encode invariants using the type system totally depends on a programmer. Experience matters here.
> Our program can't "pass through" this data from one end to an other
It's a valid point, but can be addressed by passing data as tuple (parsedData, originalData).
> What if there's a new Tag added? What if the Category tag adds a new field?
If it doesn't require changes in your code, you've modelled your domain wrong - tags should be just a Map[String, String]. If it does, you have to refactor+redeploy anyway.
> What about something like updating all updatable fields of an object
I'm not sure what exactly you meant here, but if you want to transform object in a boilerplate-free way, macroses are the answer. There is even a library for this exact purpose: https://scalalandio.github.io/chimney/! C# and Java have to resort to reflection, unfortunately.
> In 2021, I find it hard to justify using a dynamically typed language for any project that exceeds a few hundreds of lines. It's not a trade off, it's a net loss.
Only if you are skimping on tests. There's a tradeoff here - "dynamically typed" languages generally are way easier to write tests for. The expectation is that you will have plenty of them.
Given that most language's type systems are horrible (Java and C# included) I don't really think it's automatically a net gain. Haskell IS definitely a net gain, despite the friction. I'd argue that Rust is very positive too.
Performance is not dependent on the type system, it's more about language specification (some specs paint compilers into a corner) and compiler maturity. Heck, Javascript will smoke many statically typed languages and can approach even some C implementations(depending on the problem), due to the sheer amount of resources that got spent into JS VMs.
Some implementations will allow you to specify type hints which accomplish much of the same. Which is something you can do on Clojure by the way.
Automatic 'refactorings' is also something that's very language dependent. I'd argue that any Lisp-like language is way easier for machines to process than most "statically typed" languages. IDEs and debugability... have you ever used Common Lisp? I'll take a condition system over some IDE UI any day. Not to mention, there's less 'refactoring' needed.
Package management is completely unrelated to type systems.
Rust's robust package management has more to do with it being a modern implementation than with its type system. They have learned from other's mistakes.
Sure, in a _corporate_ setting, where you have little control over a project that spans hundreds of people, I think the trade-off is skewed towards the most strict implementation you can possibly think of. Not only type systems, but everything else, down to code standards (one of the reasons why I think Golang got popular).
In 2021, I would expect people to keep the distinction between languages and their implementations.
Here's what I've noticed with my tests and dynamic languages. I'll get type errors that static typing would have caught. However those errors occur in places I was missing testing of actual functionality. Had I had the functionality tests, then the type error would have been picked up by my tests. And had I just had static typing, the type system would not have been enough to prove the code actually works, so I would have needed tests anyways.
Point being, I don't really buy that a static type system saves me any time writing and maintaining tests, because type systems are totally unable to express algorithms. And with a working test suite (which you will need regardless of static vs dynamic) large refactors become just as mechanical in dynamic languages as they are in static languages.
> type systems are totally unable to express algorithms
You don't know much about types if you think that.
As for dynamic typing "helping" you to find code that you need to write tests for: There are already far more sophisticated static analysis tools to measure code coverage.
Yeah, I had a fairly large (about a year of solo dev work) app that I maintained both Clojure and F# ports of, doing a compare and contrast of the various language strengths. One day I refactored the F# to be async, a change that affected like half the codebase, but was completed pretty mechanically via changing the core lines, then following the red squigglies until everything compiled again, and it basically worked the first time. I then looked at doing the same to the Clojure code, poked at it a couple times, and that was pretty much the end of the Clojure port.
Hey, so my my career path has been C# (many years) -> F# (couple years) -> Clojure (3 months). I understand multithreading primarily through the lens of async/await, and have been having trouble fully grokking the Clojure's multithreading. One of the commandments of async/await is don't block: https://blog.stephencleary.com/2012/07/dont-block-on-async-c...
Which is why the async monad tends to infect everything. Clojure, as far as I can tell so far, doesn't support anything similar to computation expressions. So I'm guessing your "poked at it a couple times" was something like calling `pmap` and/or blocking a future? All my multithreaded Clojure code quickly blocks the thread... and I can't tell if this is idiomatic or if there's a better way.
Multi-threaded code is normally not implemented in an async style, but instead is done where each thread of execution is synchronous.
Async style comes into play generally for languages that lack real threads, or as a way to manage callbacks (even if single threaded), or in order to wait for blocking IO without the need for a real thread.
So ya, it's idiomatic to use blocking to coordinate between different threads in Clojure, same as Java.
Java decided to work on making stackful coroutines instead of stackless like C#. That requires a lot more work, but should be coming eventually to Java. At that point, your "blocking" code in Clojure will no longer block a real thread, but a lightweight fiber instead. But patience is needed for it.
In the meantime, if you're dealing with non-blocking IO that operates with callback semantics or other callback style code, what you can do in Clojure to make working with that easier is use one of:
Not even. It was opening it, looking, realizing it would take a couple weeks, and going back to F#. I did this a couple times before fully giving up.
IIRC/IIUC, Clojure's async support is closer to Go's (I've never used go), in the form of explicit channels. Though you can wrap that in a monad pretty easily, which I did for fun one day (https://gist.github.com/daxfohl/5ca4da331901596ae376). But neither option was easy to port AFAICT before giving up.
Note it's possible that porting async functionality to Clojure may have been easier that I thought at the time. Maybe adding some channels and having them do their thing could have "just worked". I was used to async requiring everything above it to be async too. But maybe channels don't require that, and you can just plop them in the low level code and it all magically works. A very brief venture into Go since then has made me wonder about that.
Yeah, quite possible. I haven't worked on the project in ~six years and lost all context, but I'd revisit it and see if perhaps there was a simple solution if any of it was still current.
My question is how does that work in a dynamically typed language? In static typed language we can know scope & type of a variable and we can't change much in runtime.
I don't pick Clojure for its dynamic typing, I pick it for other reasons. I've tried Haskell but it really doesn't seem to mesh with the way I tend to develop a program. But I would love to have more static languages with the pervasive immutability of Clojure.
It's your opinion though, there's nothing scientific about what you're saying.
Take mocking for example, in Ruby/Rails it's a breeze. In Java you need to invent a dependency injection framework (Spring) to do it.
The best response from the statically-typed world is functional programming and explicit dependencies (Haskell, OCaml, F#), which makes mocking unnecessary most of the time. OOP (Java, C#) is not the true standard for static-typing, just the most common one.
I think you are mistaken. Mocking and DI frameworks are two unrelated concepts. There is nothing in Java that forces you to use a DI framework, e.g., Spring if you want to use mocks during testing.
Let's say I have a class called User and in it a method that says the current time. So User#say_current_time
which simply accesses the Date class (it takes no arguments).
Can you show me how you would mock the current time of that method in Java?
If you want to use DI, in java 8 you could inject a java.time.Clock instance in the constructor and provide a fixed instance at the required time in your test e.g.
Instant testNow = ...
User u = new User(Clock.fixed(testNow, ZoneOffset.UTC));
u.sayCurrentTime();
although it would be better design to have sayCurrentTime take a date parameter instead of depending on an external dependency.
In my experience the need to mock out individual methods like this is an indication that the code is badly structured in the first place. The time source is effectively a global variable so in this example you'd want to pass the time as a parameter to `sayCurrentTime` and avoid the need to mock anything in the first place. A lot of C#/java codebases do seem to make excessive use of mocks and DI in this way though.
Without using a mock framework, assuming User#say_current_time isn't a private or static method then:
final Date testDate = someFixedDate;
User testUser = new User() {
@Override
Date say_current_time() {
return testDate;
}
};
If it is private and/or static, you can get around it without having to change the code, but if you own the code, you should just do that... Often the change will be as simple as replacing some method's raw usage of Date.now() with a local say_curent_time() method that uses it or some injected dependency just so you can mock Date.now() without hassle.
But your point further down that in Java you have to think about your code structure more to accommodate tests is valid. I think it's easy to drink the kool-aid and start believing that many code structuring styles that enable easier testing in Java are actually very often just better styles regardless of language, but you're not going to really see the point if you do nothing but Ruby/JS where you can get away with not doing such things for longer. Mostly it has to do with dynamic languages offering looser and later and dynamic binding than static languages (which also frequently makes them easier to refactor even if you don't have automated tools). One big exception is if your language supports multiple dispatch, a lot of super ugly Java-isms go away and you shouldn't emulate them. The book Working Effectively with Legacy Code is a good reference for what works well in Java and C++ (and similar situations in other languages), it's mostly about techniques for breaking dependencies.
I'll take clean contractual interfaces (aka actual principle of least surprise) over "I can globally change what time means with one line of code!" on large projects every time.
OK. first I could be ignorant about Java since I haven't touched it in more than a decade. Which library is doing that? And also what is mock(User.java) returning - is it an actual User instance or a stub? I want a real User instance (nothing mocked in it) with just the one method mocked.
And again if this is possible I will admit ignorance and tip my hat at the Java guys.
I think what you want is a "spy" (partial mock), not a full "mock", but yes, both are possible. You can partially mock classes, i.e., specific methods only. Syntax is almost the same, instead of mock(User.class) you write spy(User.class).
It's Mockito [1], which has been a standard for a while. There are other libraries and they use different strategies to provide this kind of functionalities (dynamic proxies, bytecode weaving, annotation processing, etc...).
In theory, I agree, but I don't think that holds terribly true in practice.
One of the ideas behind IoC frameworks (which build on top of DI) is that you could swap out implementation classes. For a great deal of software (and especially in cloud-hosted, SaaS style microservice architecture) the test stubs are the only other implementations that ever get injected.
Most code bases could ditch IoC if Java provided a language-level construct, even if that construct were only for the test harness.
Java has a mechanism, just pass alternate implemenations in constructors. If you must, a setter method. For most code you don't need to bring in the overhead of Spring, and @Autowired isn't really more convenient typing wise. Plus your unit tests become trivial, they're just POJOs with @Test annotations.
Spring is great when you need that dynamic control at runtime (especially when code dependencies are separated by modules) but you're just aping what good dynamic languages like Clojure or Common Lisp give you for free. But I can't complain too much, developing modern Java with its popular frameworks and with JRebel is getting closer to the Lisp experience every year, I'd rather have that than for Java to remain stagnate like in its 1.6/1.7 days.
The fact that there are such libraries in existence means that there is no pain associated to this particular activity. Not only do you get great mocking frameworks, they are actually very robust and benefit from static types.
Mocking dynamically typed languages is monkey patching, something that the industry has been moving away from for more than a decade. And for good reasons.
> The fact that there are such libraries in existence means that there is no pain associated to this particular activity
I can say the same about Rails + RSpec. It exists therefore it's good.
> Mocking dynamically typed languages is monkey patching, something that the industry has been moving away
That's a reach. There are millions of javascript/python/php/ruby/elixir devs that don't use types or annotations. They mock. "The industry" isn't one cohesive thing.
Maybe I've just never given it a chance, but I've never understood the appeal of being able to modify code in-memory while it's running.
I like a REPL for testing things out, or for doing quick one-off computations, but that's it. I would never want to, say, redefine a function in memory "while the code is running". Not just because of ergonomics, but because if I decide to keep that change, I now have to track down the code I typed in and manually copy it back over into my source files (assuming I can still find it at all). And if I make a series of changes over a session, the environment potentially gets more and more diverged from what's in sourced if I forget to copy any changes over. So I'd often want to re-load from scratch anyway, at least before I commit.
Am I missing something? Am I misunderstanding what people mean when they talk about coding from a REPL?
You can get an approximate taste of what it's like in plain old Java + JRebel, it's seriously about the same as trying to do it with Python + something like Flask. Start up a big application server in debug mode with Eclipse/IntelliJ, be annoyed that it takes 2+ minutes to start after everything's compiled. Now you want to work on a story, or some bug. You can make changes, save, it recompiles just that file (incremental compilation is a godsend in itself), and hotswaps it into the running application memory. No need to restart anything (with JRebel, for most common kinds of changes; without JRebel, only for some changes). It's particularly useful when you're debugging, you find the problem, change the code, and re-execute immediately to verify to yourself it's fixed.
You also can get an experience like PHP, where you just have to change and save some files, and your subsequent requests will use the new code. This is so much better than shutting down everything and restarting and is a large part of why CGI workflows dominated the web.
Common Lisp takes these experiences and dials them to 11, the whole language is built to reinforce the development style of dynamic changes, rather than an after-thought that requires a huge IDE+proprietary java agent. It's still best to use some sort of editor or IDE, and then you don't have any worry about source de-syncs -- frequently you'll make multiple changes across multiple files and then just reload the whole module and any files that changed with one function call, which you might bind to an editor shortcut, but crucially like debugging is not centrally a feature of the editor but the language; the language's plain REPL by itself is just a lot more supportive of interactive development than Python/JS/Ruby's. Clojure, and I personally think even Java with the appropriate tools, are between Python and CL for niceness of interactive development, but Clojure tends to be better than the Java IDE experience because of its other focus on immutability.
As others have mentioned - you’re probably talking about something like a python or node repl. Lisp repl development is not like python or node - you _work_ in the repl. The closer comparison might be between bash+node as a repl - up+enter and the like to rerun tests has an equivalent in clojure+IntelliJ. There’s no copy pasting but there are different key bindings.
One of the best parts about lisp style repl development is that you end up doing TDD automatically. You just redefine a function until it does what you want from sample data you pass in - without changing files or remembering how your test framework works. You can save the output of some http call in a top level variable and iterate on the code to process it into something useful. The code you evaluate lives in the file that will eventually house it anyway so it’s pretty common to just eval the entire file instead of just one function.
Since you don’t ever shut the repl down, developing huge apps is also quite pleasant. You only reload the code that you’re changing - not the rest of the infra so things like “memoize” can work in local development. That’s why it’s a bit closer to your bash shell in other languages.
If you’ve never tried it, I highly recommend trying the Clojure reloaded workflow [1] to build a web app with a db connection. You can really get into a flow building stuff instead of waiting for your migrations to run on every test boot.
Yes, what you are missing is that usually you are using an editor like emacs where you can modify a specific form and send it to the repl. That way there is no chasing back through repl history for a form to paste back into a file.
I've admittedly not played with spec, but can't you solve documenting interfaces by defining `defrecord`s ? You rarely really care about the actual types involved. You just want to know which fields you either need to provide or will recieve
Spec will give you stronger feedback than a docstring or function signature. It can tell you (in code terms, with a testable predicate) if a call to an interface wouldn't make sense.
Eg, spec can warn you when an argument doesn't make sense relative to the value of a second argument. Eg, with something like (modify-inventory {:shoes 2} :shoes -3) spec could pick up that you are about to subtract 3 from 2 and have negative shoes (impossible!) well before the function is called - so you can test elsewhere in the code using spec without having to call modify-inventory or implement specialist checking methods. And a library author can pass that information up the chain without clear English documentation and using only standard parts of the language.
You can't do that with defrecord, but it is effectively a form of documentation about how the arguments interact.
There is very little spec logic. It looks a lot like type declarations in typed languages.
It's usually outside the scope of functions, since you are likely going to want to reuse those declarations. For example, you can use spec to generate test cases for something like quick-check.
You can add pre and post conditions to clojure function's metadata that test wether the spec complies with the function's input/output.
This is a consequence of weak typing rather than dynamic typing. I appreciate that these are not precise terms, but being able to change something's type (dynamic) is different to the language just doing strange things when you combine types (weak).
You mean implicit type conversions? That's a thing you can get somewhat used to. But it throws off beginners and can introduce super weird bugs, because they hide bugs in weird ways, even if you are more experienced. Yes, I find strong typing strictly better than weak typing.
An even better example of this would be Excel, the horror stories are almost incredible.
So even if your environment is dynamic, you want clarity when you made a mistake. Handling errors gracefully and hiding them are very different things. The optimal in a dynamic world is to facilitate reasoning while not restricting expression.
It's always worth reminding folks that weak typing and implicit conversions can plague statically typed languages. C's implicit pointer array-to-pointer and pointer-type conversions are a major source of bugs for beginner and experienced programmers alike.
Agreed. I feel Lisps and SmallTalk are dynamic done right. I think the other language features that you use also influence the value from dynamic or static types. For OOP style, static types are a huge asset for refactoring and laying our architecture. On the other hand, immutable data and stateless functions (as idiomatic in clojure) make them less necessary, and also work great together with interactive development.
Not only that, Smalltalk and Lisps are languages designed with developer experience as part of the language.
You just don't get an interpreter/compiler and have to sort everything else by yourself, no, there is a full stack experience and development environment.
I keep seeing lisp people bandy about all of this design by contract/arbitrary predicate validation stuff. Can you give an example of an instance in which static types + runtime checks don't completely subsume this?
My intuition is that almost all of these methods people are talking about would have to be enforced at run-time, in which case I don't see how it's providing anything fundamentally more than writing an assertion or a conditional.
Except, of course, that specs are only tested correct, not proven correct like types would be. Types (in a reasonable static type system, not, say, C) are never wrong. In addition, specs do not compose, do they ? If you call a function g in a function f, there is no automatic check that their specs align.
Yeah, and I think this is obvious, but it certainly depends on the origin of the data being checked. We can prove the structure of “allowed” data ahead of time if we want guarantees on what’s possible inside our program. We also want a facility to check data encountered by the running program (i.e. from the user or another program.) which of course we can’t know ahead of time.
It is a design decision to be able to build a clojure system interactively while it is running, so a runtime type checker is a way for the developer to give up the safety of type constraints for this purpose—by using the same facility we already need in the real world, a way to check the structure of data we can’t anticipate.
> Types (in a reasonable static type system, not, say, C) are never wrong.
Oh man. This is the fundamental disagreement. Sure, you can have a type system that is never wrong in its own little world. But, that's not the problem. A lot of us are making a living mapping real world problems into software solutions. If that mapping is messed up (and it always is to some degree) then the formal correctness of the type system doesn't matter at all. It's like you got the wrong answer really, really right.
> A lot of us are making a living mapping real world problems into software solutions. If that mapping is messed up (and it always is to some degree) then the formal correctness of the type system doesn't matter at all.
If I'm understanding you correctly, you're saying statically typed language can't protect against design flaws, only implementation flaws. But implementation flaws are common, and statically typed languages do help to avoid those.
I'm not saying types always model your problem properly! That's not even well specified. I'm saying that "x has type foo" is never wrong if the program typechecks properly. That's totally different, and it means that you can rely on type annotations as being correct, up-to-date documentation. You can also trust that functions are never applied to the wrong number of arguments, or the wrong types; my point is that this guarantees more, in a less expressive way, than specs.
> Except, of course, that specs are only tested correct, not proven correct like types would be.
Yes this is the fundamental tradeoff. Specs et al are undoubtedly more flexible and expressive than static type systems, at the expense of some configurable error tolerance. I don't think one approach is generally better than the other, it's a question of tradeoffs between constraint complexity and confidence bounds.
Yes, I think that is one of the big weaknesses of it. You can write specs that make no sense and it will just let you. So far there is also no way to automatically check whether you are strengthening a guarantee or weaken your assumptions relative to a previous spec. In a perfect world we would have this in my opinion.
I feel like there's a missing axis in the static/dynamic debate: the language's information model.
In an OOP language, types are hugely important, because the types let you know the object's ad-hoc API. OOP types are incredibly complicated.
In lisps, and Clojure in particular, your information model is scalars, lists, and maps. These are fully generic structures whose API is the standard Clojure lib. This means that its both far easier to keep the flow of data through your program in your head.
This gives you a 2x2 matrix to sort languages into, static vs dynamic, and OOP vs value based.
* OOP x static works thanks to awesome IDE tooling enabled by static typing
* value x static works due to powerful type systems
* value x dynamic works due to powerful generic APIs
* OOP x dynamic is a dumpster fire of trying to figure out what object you're dealing with at any given time (looking right at you Python and Ruby)
They don't make these impossible, they typically just don't let you express these within the type system and they typically don't let you not specify your types.
I should have made clear that I'm emphasizing the advantages of being dynamic to describe and check the shape of your data to the degree of your choosing. Static typing is very powerful and useful, but writing dynamic code interactively is not just "woopdiedoo" is kind of the point I wanted to make without being overzealous/ignorant.
That largely depends on the type system. Languages like Haskell and Scala which have much more powerful type systems than C/Java/Go/etc absolutely do allow you to do those sorts of things. It is a bit harder to wrap your head around to be sure and there are some rough edges, but once you get the hang of it you can get the benefits of static typing with the flexibility of dynamic typing. See https://github.com/milessabin/shapeless or a project that I've been working on a lot lately https://github.com/zio/zio-schema.
Well it does include that kind of behaviour but it's quite a bit more than just that. E.g. you could express something like "the parameter must be a date within the next 5 business days" - there's no static restriction. I'm not necesarily saying you should but just to give an illustrative example that there's less restrictions on your freedom to express what you need than in a static system.
>> types are isomorphic with schemas
I don't think that's a good way to think of this, you're imagining a rigid 1:1 tie of data and spec yet i could swap out your spec for my spec so that would be 1:n but those specs may make sense to compose in other data use cases so really it's m:n rather than 1:1
> E.g. you could express something like "the parameter must be a date within the next 5 business days" - there's no static restriction
Hm, I don't follow. If I were to write this in F#, there would be a type `Within5BusinessDays` with a private constructor that exposes one function/method `tryCreate` which returns a discriminated union: either an `Ok` of the `Within5BusinessDays` type, or an `Error` type with some error message. Once I have the type, I can then compose it with whatever and send it wherever and since F# records are immutable, I won't have to worry about invariants not holding. And since it's a type, I have the compiler/type system on my side to help with correctness.
(Side note, this is a bad example since the type can become invalid after literally 1 second... but since Clojure has the same problem I'm just running with it.)
I'm still learning Clojure (only a few months into it), but if I were to to write a spec, I'd have to specify what to do do if the spec failed to conform - same as returning the `Error` case in F#.
> i could swap out your spec for my spec so that would be 1:n but those specs may make sense to compose in other data use cases so really it's m:n rather than 1:1
Sorry, but I'm still not following - I believe you can do the same with types, especially if the type system support generics.
> If I were to write this in F#, there would be a type `Within5BusinessDays`
That’s not really the same thing - it’sa valid alternative approach but you’ve lost the benefits of a simple date - from (de)serialisation to the rich support for simple date types in libraries and other functions, the simple at-a-glance understanding that future readers could enjoy. Now the concept of date has been complected with some other niche concern.
> the type can become invalid after literally 1 second
Every system I’ve ever seen that has the concept of a business date strictly doesn’t derive it from wall clock date. E.g. it’s common that business date would be rolled at some convenient time (and most often not midnight) so you’d be free to ensure no impacts possible from the date roll.
>> I believe you can do the same with types, especially if the type system support generics
You can do something similar but you’ll need to change the system’s code.
It would be almost like gradual typing, except you could further choose to turn it off or to substitute your own types / schema without making changes to the system / code.
It’s quite a lot more flexible.
(Apols for slow reply - i 1up’d your reply earlier when i saw it but couldn’t reply then)
Right! I made the dumb, typical error to write: "You simply do not have that in a static world." When I should have written: "This type of expressiveness is not available in mainstream statically typed languages".
With "freely composable" I mean that you can program with these schemas as they are just data structures and you only specify the things you want to specify. Both advantage and the disadvantage is that this is dynamic.
I tried to use Clojure but what put me of was that simple mistakes like missing argument or wrongly closed bracket didn't alert me until I tried running the program and then gave me just some java stack spat out by jvm running Clojure compiler on my program.
> didn't alert me until I tried running the program
That's because that's not how Clojure developers normally work. You don't do changes and then "run the program". You start your REPL and send expressions from your editor to the REPL after you've made a change you're not sure about. So you'd discover the missing argument when you call the function, directly after writing it.
Interesting. How exactly that looks? Do you have files opened in your editor, change them then go into previously opened repl, and just call the functions and the new version of those function runs?
Thanks to the dynamic nature of Clojure programs, experienced Clojure developers use the REPL-driven development workflow as demonstrated in this video [1].
From what I understand, instead of writing the file and running the file you write separate statements in the file and evaluate each of them in the repl (like with "Do it" in Smalltalk).
So what you get, after running the file afterwards from clean state might be different than the result of your selective separate manual evaluations.
This looks like exactly the opposite of the F5 workflow in the browser where you can run your program from clean state with single keypress.
I haven't watched the video till the end though maybe there's a single key that restarts the repl and runs the files from clean state here too.
At first glance you could have the same workflow with JS, but there's not much need for it because JS VMs restart very quickly and also you'd need to code in JS in very particular style, avoiding passing function and class "pointers" around and avoid keeping them in variables. I guess clojure just doesn't do that very often and just refers to functions through their global identifiers, and if that's not enough, even through symbols (like passing the #'app in this video instead of just app).
That's right. You typically would have your text editor/ide open, and the process you're developing would expose a repl port which your editor can connect to. As you edit the source code, that will automatically update the code running in the process you're debugging. See this demo of developing a ClojureScript React Native mobile app published yesterday: https://youtu.be/3HxVMGaiZbc?t=1724
> We tried VisualVM but since Clojure memory consists mostly of primitives (Strings, Integers etc) it was very hard to understand which data of the application is being accumulated and why.
I was going to suggest this -- inside of VisualVM, you can right-click a process and then press "Start JFR"
Then wait a bit, right click it again, and select "Dump JFR"
What you get is a Flight Record dump that contains profiling information you can view that's more comprehensive than any language I've ever seen.
I used this for the first time the other day and felt like my life has been changed.
Specifically, if you want to see where the application is spending it's time and in what callstacks, you can use the CPU profiling and expand the threads -- they contain callstacks with timing
There's some screenshots in an issue I filed here showing this if anyone if curious what it looks like:
Walmart Labs was a step in this direction.. but we need some big companies to standardize around Clojure to jumpstart the ecosystem of knowledge, libraries, talent, etc. I’ve spoken to engineering hiring managers at fairly big companies and they’re not willing to shift to a niche language based only on technical merits but without a strong ecosystem.
If we don’t get some big companies to take on this roll the language is going nowhere.
I’m saying this because I’m a huge fan of Clojure (as a syntax and language, not crazy about the runtime characteristics) and I hope I get the opportunity to use it.
What does Google Trends have to do with a programming language? PHP is trending, and Clojure is not, perhaps because Clojure gives you a lot fewer reasons to google stuff up?
I myself rarely use Google to find a solution to a problem, and certainly almost never have to google shit like: "how to open a file in Clojure"...
I started working professionally with Clojure earlier this year and this article rings true. I think the article leaves out a fourth downside to running on the JVM: cryptic stack traces. Clojure will often throw Java errors when you do something wrong in Clojure. It's a bit of a pain to reason about what part of your Clojure code this Java error relates to, especially when just starting out.
I work at Ladder [0], and almost everything is done in Clojure/ClojureScript here. I had no previous experience in Clojure – Ladder ramps you if you haven't used it before. My interview was in Python. We're currently hiring senior engineers, no Clojure experience necessary [1].
To be fair, this is not unique to Clojure. You need to deal with stack traces no matter what as long as you're using any programming language that targets the JVM (even statically type-checked languages like Scala). There are some great articles [1][2] that discuss various simple techniques helpful for debugging and dealing with stack traces.
I've never really had a problem with stack traces in Scala. Every once in a while you hit a cryptic one that's buried in Java library code, but for the most part they're runtime errors that are due to incompletely tested code or some kind of handled error with a very specific message.
> ... and the question regarding choosing Clojure as our main programming language rose over and over again
If I find myself having to repeat myself justifying a certain decision time and time again, it's an indicator that the decision needs to be revised to be something which is a more intuitive fit for the organization.
That's not a good indication that the decision was or was not correct. Only that it currently runs against whatever the established practice is. Sometimes "the way things have always been done" is just wrong.
This is unlikely to be the case in the choice of programming languages. Some may be a bad fit, some may have ecosystems that are unpleasant to use, but it's generally not the biggest problem an organization will have.
Not really; it's like stoplights. You're going to be interrupted and therefore notice the red lights, and just sail easily through and thus not notice the green lights. Likewise, you're going to notice the pain points, but need to take a minute to reflect to notice the benefits.
Really, if repeating the same justifications convinces people, then the problem isn't the justifications.
I donno why you're being downvoted, it's a questionable decision and probably the company would have been better off with Python/PHP/Node. Hiring and onboarding are extremely important for a startup. You know what else? Finding answers to common questions on Google/Stackoverflow; I am now working with Ember and can tell you guys you take a 50% productivity hit by using a tool that's obscure on Google. Sure once you become super familiar with a tool that matters less, but that takes time. Much more time. React/Angular may be an inferior tool to Ember but the fact that you can get answers to almost any question is priceless. The community size is super important. The frameworks are super important (is there a Closure equivalent to Rails/Django/Laravel in community size, in battle testedness? I really doubt it).
That being said, I salute these brave companies for sticking to these obscure languages. Do we want to live in a world where there's only 3 languages to do everything? Even 10 sounds boring. Hell, even a fantastic tool like Ruby is considered Niche in certain parts of the world. I don't want a world without Ruby so I don't want a world without Closure.
> Hiring and onboarding are extremely important for a startup.
If you're a small company, you usually cannot afford to hire "mediocre" talent. It is much more expensive to undo the crapola they'd implement. Trying to hire those who are at least interested in learning and using languages like Clojure, Rust, Haskell, Elixir, Elm, etc., is a very good quality filter. ROI from hiring a smaller number of Clojure devs, rather than a few more "regular" engineers - is much higher.
> Finding answers to common questions on Google/Stackoverflow;
Clojure gives you far fewer reasons for Googling things than other language ecosystems. It is dense language and inspires you to write smaller functions, decreasing the surface area for the problem. Most of the time, asking questions in Clojurians Slack sends you halfway through the solution.
> I salute these brave companies for sticking to these obscure languages
They do not choose Clojure for the shtick; Clojure is a highly pragmatic and immensely productive instrument. There are many "success stories" with small and medium-sized companies. A few large companies like Cisco, Apple, Walmart, et al., actively develop in Clojure.
The same can be said about the engineers. They don't choose Clojure because "they hate Java". You can check any Clojure surveys of the past. Most Clojure engineers are experienced and "tired" developers. Seasoned hackers who have seen the action. For most of them - Clojure is a deliberate choice. Many of them landed in it after trying various other alternatives.
Alternatively, you could document the thought process that lead up to the decision and you can point the unenlightened to the documentation instead of having to repeat yourself.
EdwardDiego | 4 years ago
> An incoming HTTP request? it is a plain Clojure dictionary.
I learned to code in Python. Loved it. Dynamically typed dicts up the wazoo!
Then I learned why I prefer actual types. Because then when I read code, I don't have to read the code that populates the dicts to understand what fields exist.
[Deleted] | 4 years ago
robertlagrant | 4 years ago
I agree. This doesn't seem much different to saying they're all objects. You still need to know what to expect inside the dictionary.
goatlover | 4 years ago
The difference being that objects have a class where you can look to see what fields it specifies.
robertlagrant | 4 years ago
Sure, depending on the language. What I mean is having dictionaries doesn't mean you don't have to learn schemas.
dan-robertson | 4 years ago
Java doesn’t really have a nice interface for interacting with objects in general. Closure does have a nice interface for interacting with dictionaries. They have namespaces keyword symbols for keys which are much more ergonomic than typing strings, and they have lots of functions for modifying dictionaries. I think the big difference is in the philosophy of what the language thinks data is, and how the world ought to be modelled.
lvh | 4 years ago
The two are not mutually exclusive. Clojure has namespaced keywords and specs[0] to cover that. (There is also the third-party malli, which takes a slightly different appproach.)
The advantage is that maps are extensible. So, you can have middleware that e.g. checks authentication and authorization, adds keys to the map, that later code can check it directly. Namespacing guarantees nobody stomps on anyone else's feet. Spec/malli and friends tell you what to expect at those keys. You can sort of do the same thing in some other programming languages, but generally you're missing one of 1) typechecking 2) namespacing 3) convenience.
[0]: spec-ulation keynote from a few years ago does a good job explaining the tradeoffs; https://www.youtube.com/watch?v=oyLBGkS5ICk
kitd | 4 years ago
Yeah, he mentions that later on as a drawback
dan-robertson | 4 years ago
Question: 1. Can a GET request have a non-empty request body?
2. Assuming you don’t know the answer to that question, will the type system you use be able to tell you the answer to that question?
This is a pretty simple constraint one might want (a constraint that only certain requests have a body) but already a lot of static type systems (e.g. the C type system) cannot express and check it. If you can express that constraint, is it still easy to have a single function to inspect headers on any request? What about changing that constraint in the type system when you reread the spec? Is it easy?
The point isn’t that type systems are pointless but that they are different and one should focus on what the type system can do for you, and at what cost.
lkitching | 4 years ago
Any statically-typed language with generics can express that by parameterising the request type with the body type. A bodiless request is then just Request[Nothing] (or Request[Unit] if your type system doesn't have a bottom type). Accessing the headers just requires an interface which all static languages should be able to express.
dan-robertson | 4 years ago
(1) note that “statically-typed language with generics” excludes a lot of statically typed languages, including C and Go (at least pre generics).
(2) this misses the meat of the question which is how to express that (eg) a GET request doesn’t come with a body and a POST request does. I suppose that you’re suggesting that one registers a url handler with a method type and that forces the handler to accept responses of a certain type. Or perhaps you are implicitly allowing for sun types (which aren’t a thing in many static type systems.)
(3) even in C++, isn’t this suggestion hard to work with. That is, isn’t it annoying to write a program which works for any request whether or not it has a body because the type of the body must be a template parameter that adds templates to the type of every method which is generic to it. But maybe that is ok or I just don’t understand C++.
xapata | 4 years ago
How about values restricted to identifiers currently in the database table? There's always something the type system can't do.
[Deleted] | 4 years ago
jolux | 4 years ago
F# has a feature called type providers that make this sort of bookkeeping between the database and the code less tedious, but even if you mess it up, static typing still gives you more safety than dynamic. If your code blew up because it should have accepted an identifier it didn’t, you know that the code has not been written to handle that case and can fix it. Alternatively, you can just choose to ignore this, and do what a dynamic language does. There is nothing stopping you from being dynamic in a static language, passing everything around as a map, etc.
dharmaturtle | 4 years ago
A demo of a SQL type provider in action: https://youtu.be/RK3IGYNZDPA?t=2539
It requires a bit of elbow grease to make it work with a CICD system... but it works :D
xapata | 4 years ago
That's nifty.
twic | 4 years ago
1. Yes. It's weird, but it's legal HTTP.
2. Sure. The request type has a body property.
dan-robertson | 4 years ago
Does “the request type has a body property” actually imply (1) though? In a language like C or C++ or Java, you could have a protocol like “body is always null on GET requests.” The question isn’t really about HTTP, that was just an easy-to-reach-for example, it is really about what having explicit types allows one to deduce about a program.
taeric | 4 years ago
To be fair, an incoming request is, almost by definition, dynamic. It makes sense to have that as a map, since the main sensible thing to do on receipt is validation/inspection.
Granted, you may have a framework do a fair bit of that. Depends how much you want between receipt of the request and code you directly control.
Zababa | 4 years ago
Usually the approach in a statically-typed language is to transform your dynamic request into something that you know through parsing instead of validation. Here's a great article about this: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va....
taeric | 4 years ago
That is a valid approach in any language. Static or not. Doesn't change my point that heavily. And it is all too possible to pick a bad parsing/binding language such that protocol changes in the request are now foot guns.
Zababa | 4 years ago
That's true, but static languages are not worse at handling dynamic data. From the same author: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ....
taeric | 4 years ago
To an extent, I agree. I'm just pointing out that this is a bit of a bad example. I want there to be dynamic inspection of input.
That said, maps as the only tool is clearly messy. And is a straw man.
kingdomcome50 | 4 years ago
This is the second time I've seen the link above. And while I agree with the premise, the author clearly does not understand how to properly use the `Maybe` monad (a term that does not make an appearance!).
There is little use in wrapping a call in `Maybe` to then immediately unwrap the result on the next line. Doing so isn't really using the construct... One would expect the lines following the creation of `Maybe` to bind calls through the monad.
In the end I see almost no meaningful difference between their "Paying it forward" example and simply utilizing an `if` to check the result and throw. In essence the author is using a parse and validate approach!
travv0 | 4 years ago
Lexi absolutely understands how to properly use the Maybe monad. What you're saying to do here is the exact opposite of what this post is advocating for. You're talking about pushing the handling of the Maybe till later and the post is all about the advantages of handling it upfront and not having to worry about it anymore. You might want to read it one more time.
kingdomcome50 | 4 years ago
I understand. But what is purpose of `Maybe`? The reason one would reach to the above construct is precisely to offload (pushing to later) the handling of a value that may (or may not) be present at runtime such that a developer can write code assuming the value is always present and ignore the `Nothing` case.
Sure you can unwrap it right away, but that isn't necessary because you could also just "bind" the next function call to the monad (which is more idiomatic to the construct). You never have to worry about that value in this case because... well... that's the benefit of using `Maybe`.
I'm not super familiar with Haskell, but my sense is that the author is trying more to please the compiler (at a specific point in the program!) than simplify the logic. That is, they want a concrete value (`configDirs`) to exist in the body of `main` more than they want the cleanest representation of the problem in code.
travv0 | 4 years ago
> But what is purpose of `Maybe`?
In this case, it's to provide a better error message in case there's an empty list than `fromList` would provide.
> You never have to worry about that value in this case because... well... that's the benefit of using `Maybe`.
But you do, your entire program doesn't live in `Maybe` so at some point you have to check whether it's `Just a` or `Nothing`. Once again, the whole point of the post is to argue that getting out of the `Maybe` as close to parsing time as possible is preferable so you have a more specific type to work with after that. You also see right away what didn't parse instead of just knowing that something didn't parse, which is what would happen if you stayed in the `Maybe` monad for all your parsing.
kingdomcome50 | 4 years ago
> your entire program doesn't live in `Maybe`
Well... if your entire program is dependent on some input that may or may not exist at runtime... then it kind of does live in `Maybe`.
I have no issue with unwrapping a `Maybe` to throw an exception. But I do find it a bit ironic that the post is about parsing instead of validating, that the perfect construct is right there to exemplify how it could be done, but the author then chooses to eschew it and instead show examples of how validation could look.
The body of `main`, for example, could be refactored to something like:
Which actually shows how `Maybe` can be used to simplify the system. If you want to unwrap the maybe at this point to throw, go for it! But the above is a much cleaner representation of the program than what author is trying to do (it's crystal clear how the cache might get initialized). I would expect "Parse don't validate" to be about how useful `Maybe` is to combine parsing logic into a functional flow vs. how validation leads to an ugly procedural approach.garethrowlands | 4 years ago
I think you're referring to this part of the `getConfigurationDirectories` action, which has type `IO (NonEmpty FilePath)`:
The "meaningful difference" you're looking for is the type of `getConfigurationDirectories`. The previous version had type `IO [FilePath] `, which _doesn't_ guarantee any configuration directories at all. It did indeed check the results and throw. But it doesn't guarantee that all the `[FilePath]` values in the program have been checked. There are neither tests nor proofs in this code. In contrast, with the revised version, you can be certain anywhere you see a `NonEmpty FilePath` it is indeed non-empty.The code I've quoted that checks which case we have, is the only place that needs to handle that `Maybe`. Or maybe `main`, if we want to be more graceful. The author (I wouldn't say I know her but I know that much) does know how to chain maybes with bind but it's not necessary in this example code.
kingdomcome50 | 4 years ago
My point is that if you are not chaining `Maybe` then the utility of employing the construct is unobserved. The entire purpose of using `Maybe` is to relieve the client from the need to make checks at every call for a value that may (or may not) exist. If you intend to immediately "break out" of the monad and (even more specifically) throw an error, you might as well just use an `if`.
I'm sure `main` could be written to "bind"/"map" `getConfigurationDirectories` with `nonEmpty`, `head`, and `initializeCache` in a way that puts the `throw` at the top-level (of course the above implementations may need to change as well). Unfortunately I'm not familiar enough with Haskell to illustrate it myself.
lkitching | 4 years ago
The purpose of Maybe is to explicitly represent the possible non-existence of a value which in Haskell is the only option since there's no null value which inhabits every type. The existence of the monad instance is convenient but it's not fundamental. The type of getConfigurationDirectories could be changed to MaybeT IO (NonEmpty FilePath) to avoid the match but I don't think it would make such a small example clearer.
kingdomcome50 | 4 years ago
There are numerous ways to redesign the function signatures, but I would imagine the simplest would be (again, idk Haskell syntax):
Notice `nonEmpty` isn't really necessary because `head` could to the work. The above could be chained into a single, cohesive stack of calls where the result of each is piped through the appropriate `Maybe` method into the next call in a point-free style. I cannot imagine how this wouldn't be clearer. e.g: That's the whole thing. Crystal clear. The big takeaway of "Parse don't validate" should be about the predominant use of the `Maybe` monad as a construct to make "parsing" as ergonomic as possible! Each function that returns `Maybe` can be understood as a "parser" that, of course, can be elegantly combined to achieve your result.My critique is exactly that unwrapping the `Maybe` immediately in order to throw an exception is kind of the worst of both worlds. I mentioned this in a sibling comment, but my sense is that the author is more concerned with have a concrete value (`configDirs`) available in the scope of `main` than best-representing the solution to the problem in code. It is a shame because I agree with the thesis.
lkitching | 4 years ago
On the contrary the The NonEmpty type is fundamental to the approach in that example since it contains in the type the property being checked dynamically (that the list is non-empty). The nonEmpty function is a simple example of the 'parse don't validate' approach since it goes from a broader to a more restricted type, along with the possibility of failure if the constraint was not satisfied. The restriction on the NonEmpty type is what allows NonEmpty.head to return an a instead of a (Maybe a) and thus avoid the redundant check in the second example. The nonEmpty in your alternative implementation is only validating not parsing since after checking the input list is non-empty, it immediately discards the information in the return type. This forces the user to deal with a Nothing result from head that can never happen. Attempting to clean the code up by propagating Nothing values using bind is just hiding the problem that the validating approach avoids entirely.
kingdomcome50 | 4 years ago
You are misunderstanding the system. You can organize the logic into whatever containers you want, but the essence of the system cannot be changed.
You are already handling a `Maybe` type because it's possible for your input to not exist. Because the first implementation of `head` also returns a `Maybe`, it is possible to "bind" them together (I'm leaving out `IO` because I am both unsure of the syntax[0] and it is immaterial to the example):
[0] I have never written Haskell, so the above is my best-guess at the syntax given the snippets available (and no extra research)The two functions `head` and `getConfDirs` are "parsers" because they both return `Maybe`. Contrary to
> Returning Maybe is undoubtably convenient when we’re implementing head. However, it becomes significantly less convenient when we want to actually use it!
It is trivial to use a reference to `Maybe` because it is a monad that it is specifically designed to be used more conveniently than the alternative approaches in the case when a value may (or may not) exist.
dharmaturtle | 4 years ago
You might try re-reading it with some charity - the example's purpose isn't to teach the `Maybe` monad, but to remove the redundant check. To go into what `bind` does would be a diversion from the main topic (parsing vs validating).
FWIW SPJ has called this blog's author a "genius" so... I think they do know how `Maybe` works. https://gitlab.haskell.org/ghc/ghc/-/issues/18044#note_26617...
kingdomcome50 | 4 years ago
But `Maybe` is specifically designed to remove redundant checks for a value that may (or may not) be present! That's the whole point of the monad! It seems rather unfortunate this isn't highlighted (or at least illustrated) doesn't it?
I generally agree with the premise of the post.
fmakunbound | 4 years ago
This is one of those self-inflicted Clojure problems. In Common Lisp you might use an alist or a plist for small things, but you'd definitely reach for CLOS classes for things that had relationships to other things and things that had greater complexity.
IIRC, the preference for complecting things via maps, and then beating back the hordes of problems with that via clojure.spec.alpha (alpha2?) is a Hickey preference. I don't recall exactly why.
blacktriangle | 4 years ago
No source to back this up, but my guess is that Clojure was driven by the need to interopt with Java so is to not get kicked out of production. This meant absorbing the Java object model. Shipping a language with both Java objects and CLOS and making them both play nice together sounds like a nightmare.
joncampbelldev | 4 years ago
This comment helpfully explains many of the reasons Rich had for choosing immutable, persistent, generic data structures as the core information model in clojure (instead of concrete objects / classes): https://news.ycombinator.com/item?id=28041219
Not wanting to misquote the above / Rich himself I would TLDR it to:
- flexibility of data manipulation
- resilience in the face of a changing outside world
- ease of handling partial data or a changing subset of data as it flows through your program
Please note that no one (I hope) is saying that the above things are impossible or even necessarily difficult with static typing / OOP. However myself and other clojurists at least find the tradeoff of dynamic typing + generic maps in clojure to be a net positive especially when doing information heavy programming (e.g. most business applications)
tragomaskhalos | 4 years ago
Namedtuples FTW! A de-facto immutable dict with the keys listed right there in the definition to obviate all the usage head-scratching. Then, if you need more functionality (eg factory functions to fill in sensible defaults), you can just subclass it.
TBH I've never understood the attraction of the untyped dict beyond simple one-off hackups (and even there namedtuples are preferable), because like you say you typically have no idea what's supposed to be in there.
lmilcin | 4 years ago
> Pure functions make code design easier: In fact, there’s very little design to be done when your codebase consists mostly of pure functions.
Ummm... I am a little bit fearful about your codebase.
If you don't see the need for designing your FP system it probably mostly means it is being designed ad hoc rather than explicitly.
If you are trying to compare to OOP system done right, you will notice that this includes a lot of work in identifying domain model of your problem, discovering names for various things your application operates on, and so on. Just because you elect to not do all of this doesn't mean the problem vanishes, it most likely is just shifted to some form of technical debt.
> Clojure is a dynamic language which has its advantages but not once I stumbled upon a function that received a dictionary argument and I found myself spending a lot of time to find out what keys it holds.
Dynamic typing is a tradeoff which you have to be very keenly aware of if you want to design a non-trivial system in a dynamically typed language.
It is not a problem with Clojure, it is just a property of all dynamically-typed languages.
dmitriid | 4 years ago
One thing I don't like about all articles on clojure is that basically all of them say: ah, it's just like lisp with lists `(an (example of) (a list))` with vectors `[1 2 3]` thrown in. So easy!
But then you get to Clojure proper, and you run into additional syntax that either convention or functions/macros that look like additional syntax.
Ok, granted, -> and ->> are easy to reason about (though they look like additional syntax).
But then there's entirely ungooglable ^ that I see in code from time to time. Or the convention (?) that call methods on Java code (?) with a `.-`
Or atoms defined with @ and dereferenced with *
Or the { :key value } structure
There's way more syntax (or things that can be perceived as syntax, especially to beginners) in Clojure than the articles pretend there is.
ronnier | 4 years ago
Single engineers will pick clojure at companies , build a project in it, later that engineer will move on, now nobody can maintain this code so it’s rewritten in some normal language. I’ve seen that happen a few times. That code is hard to read and understand. This is why clojure will remain niche.
achikin | 4 years ago
It could have been Go and Java programmer trying to understand it. Or it could have been some clumsy tool written in node which Go programmer finds hard to read and understand. Clojure's main advantage is that you can you can learn it very very quickly up to the point when you understand most of the code, the language is very very small compared to "five main languages".
outworlder | 4 years ago
> Single engineers will pick clojure at companies , build a project in it, later that engineer will move on, now nobody can maintain this code so it’s rewritten in some normal language
"Normal language"?
You mean, whatever language is most popular at the company. What's "normal" at one would be completely alien at another. Even things like Java. If you don't have anything in the Java ecosystem, the oddball Java app will be alien and will likely get rewritten into something else.
The reason Clojure remains niche is that some people somehow think it's not a "normal" language, for whatever reason.
taeric | 4 years ago
That is possible with all languages. I've seen java, scala, clojure, perl, python, etc.
Usually this is made worse by bespoke build tools and optimizations that make the system punishing to pick up.
sramsay | 4 years ago
You've seen a case where someone wrote something in Python that later devs could not understand and then rewrote it in . . . what? And you've seen that with Java?
There's a big difference between a developer going off and writing something in one of the top five most used languages in the world and doing so in Scala.
taeric | 4 years ago
Yes. I've seen and contributed to dumpster fires in all of those languages. I would love to say it was all some rogue developer that crapped on things, but it is often just new developers. The more, the more damage.
lostcolony | 4 years ago
Both are strange and alien to Javascript developers, who can be full stack.
Python may seem simple once you know it, but going in blind there's plenty of traps to bite you. Significant whitespace for one.
chrsig | 4 years ago
I think there's two different issues:
1. picking a language/tool that a company doesn't have personnel with experience using it
2. picking a language/tool that is esoteric, which generally implies #1 as well.
#1 on its own isn't great, but generally when sticking in the java/python/ruby/javascript/php/etc...mainstream languages, there's a lot more documentation, and there's a higher chance that _someone_ in the company will have some familiarity. If nothing else, it'd be easier to hire a replacement for.
lostcolony | 4 years ago
A higher chance, yes, but it doesn't matter much; what is tricky with most applications is the domain. Certainly, it's faster to go learn a language than to learn a new domain. To that end, you can get the whole team trained faster in a language than you can hire someone with experience and train them to the domain.
joelbluminator | 4 years ago
> Certainly, it's faster to go learn a language than to learn a new domain.
It's not only the language but the framework. For example I know javascript well enough but I now am quite a noob with Ember in my new role. I would say the framework is just as important as the language, at least when doing web development.
chrsig | 4 years ago
You're kind of reinforcing the point though -- now you've got a whole team distracted by picking up a new language....why? how is it a good use of anyone's time? And it'll be a perennial training issue in the case of an esoteric language, because those team members will eventually turn over as well, meaning that you don't get to avoid either hiring or training a new person on it.
If it's just one component, implemented by a single dev, it really can make more sense to understand what it does and rewrite it in a language that's common in the company.
lostcolony | 4 years ago
I'm not advocating NOT rewriting it. I'm just saying, back to the great grandparent's point, that the issue is a dev went rogue, NOT the language the rogue dev chose. The difficulty is the same regardless of the language the rogue dev chose; it's not that they picked Clojure, it's that they picked a language there was no organizational adoption of.
agumonkey | 4 years ago
is it really hard to read (could be) or is it just that the average coder never saw lisp or sml and doesn't want to bother bearing the responsibility to learn something alien on duty ?
mollusk | 4 years ago
You need a team that wants to use Clojure. I wrote Clojure professionally for 2 years, and everyone at the company was excited about it and sold on the language. Even after 3-5 years of programming in it. Now, at a different place, we write in a different language, and even though I still love Clojure, I'm not gonna write some project in it, even if Clojure might suit it so well, because I know these people are sold on different language, and I'm not going to preach and I'm not going to make their lives more difficult by having to maintain some obscure codebase.
lvh | 4 years ago
Minor point of order about the atoms: they're not defined with @ nor derefd with . If you're referring to earmuffs* that's convention not syntax (specifically for dynamically scoped variables, which could be atoms or anything else), and @ is indeed deref. (More specifically @x is a reader macro ish that expands to literally `(deref x)`.)
dmitriid | 4 years ago
Thank you! I never seem to remember this (but I don't use Clojure, so it's not an ingrained knowledge)
girishso | 4 years ago
Agreed. These days I'm really fascinated by clojure and trying to learn clojure. Other than the project setup and repl and the editor (which I had considered), these weird characters are throwing me off.
What clojure really needs is some kind of opinionated framework or starter template, something like create-react-app. That has all these things figured out so a beginner like me can start playing with actual clojure, which documents all the steps to setup the repl and editor and what not. The last time I asked for this I was told about lein templates, they help but there's no documentation to go with those.
There needs to be some push from the top level. create-react-app was produced by facebook. Elm reactor (which lets you just create a .elm file and play with elm) was created by Evan the language creator himself.
tldr: There's a huge barrier to start playing with clojure that needs to come down and the push needs happen from the top level.
uDontKnowMe | 4 years ago
There is the widely used Luminus framework https://luminusweb.com/
girishso | 4 years ago
Yes, of course and I've got the book as well. The problem with the book is I got stuck on the very first code example in the book. I know there's a forum for the book where (hopefully) I can get my query answered.
My point is: these are all individual attempts (the book i mean) and there will always be something on page xyz broken and it can't be solved by individuals. To solve these problems, there needs to be constant time and money investment from someone serious (like facebook in case of create-elm-app).
uDontKnowMe | 4 years ago
Yes I agree there is a problem of a lack of institutional funding in the Clojure world. Luminus is a great tool but it is a bit sad that it is arguably the most production-ready web toolkit in the ecosystem and it is mostly the work of a single person.
There is some community effort to better fund the core infrastructure in Clojure through https://www.clojuriststogether.org/, hopefully they can continue to attract more funding developers and companies.
In general a lot of these issues could be alleviated if the community was just in general larger with more contributors. I think the Clojure community is quite welcoming to newbies in the sense that people are quite responsive, kind and helpful around the internet, in Clojurians Slack (try asking there btw, if you haven't yet and are still stuck at the start of the book), etc. But in other ways people seem averse to criticism or suggestions from outsiders. I think the Clojure world needs to do a bit of self reflection to understand why adoption is so low right now and honestly consider what needs to change to attract more developers and contributors.
cr__ | 4 years ago
Not sure how you’re supposed to find this page, but it’s pretty useful: https://clojure.org/guides/weird_characters
dmitriid | 4 years ago
Nice! I missed it (or it didn't exist) when I last looked at Clojure a few years back
roenxi | 4 years ago
You missed it, it has been there forever. But it says good and bad things about Clojure that its reference documentation is one of its weakest points.
The Guide/Reference split obscures a lot of information (do I want guidance on Deps & CLI or do I want reference on Deps & CLI?) and the guides where that gem is hidden randomly mix advanced topics (eg, how to set up generative testing), beginner topics (how to read Clojure code) and library author topics (eg, Reader Conditionals).
When you think about it, there is nearly no trigger to look at the guides when the information you need is there. Clojure is a weird mix of both well documented and terribly documented. All the facts are on the website, very few of them are accessible when required. The people who make it past that gauntlet are rewarded by getting to use Clojure.
dmitriid | 4 years ago
In my case it was even worse, as I started with ClojureScript, and official documentation was simply abysmal then.
hcarvalhoalves | 4 years ago
Finally,
is the same as and the `*` means nothing to the language – any character is valid on symbol names, the author just chose an asterisk.Most of what you believe to be syntax are convenience "reader macros" (https://clojure.org/reference/reader), and you can extend with your own. You can write the same code without any of it, but then you'll have more "redundant" parenthesis.
dmitriid | 4 years ago
> Most of what you believe to be syntax are convenience "reader macros"
And yet, you need to know what all those ASCII symbols mean, where they are used, and they are indistinguishable from syntax.
Moreover, even Clojure documentation calls them syntax. A sibling comment provided a wonderful link: https://clojure.org/guides/weird_characters
MeteorMarc | 4 years ago
What build tools do you use, maven?
finalfantasia | 4 years ago
Clojure developers tend to choose the official Clojure CLI tools [1] for new projects these days.
[1] https://clojure.org/guides/deps_and_cli
tribaal | 4 years ago
Not the author, but most clojure projects use leiningen to build and distribute projects (https://leiningen.org/)
This seems to be the case for the author's open-source work (https://github.com/nanit/kubernetes-custom-hpa/blob/master/a...)
evanspa | 4 years ago
Great article, love Clojure. Was trying to figure out what Nanit does. Might want to consider putting a link to the Nanit homepage on your engineering page. When just typed in nanit.com and saw the baby monitor tech, I thought maybe I went to the wrong place, until I saw the logos matched. Anyway, good read, but please put a link to your home page on your engineering site, or, put a 1 liner in the opening of your blog giving context to what your company does.
didibus | 4 years ago
Being able to keep track of what data was where is the initial bump I had as well when learning Clojure. Unlike the author, I personally got used to it, and generally don't struggle with it anymore, but part of that is learning good habits on your code base where you make judicious use of names, doc-string, destructuring and have a well defined data model using records or Spec or Schema, etc.
The other one is just getting good at the REPL and inspecting the implementation for functions to quickly see what keys and all they make use of.
Something the article didn't really cover either is that it's not really the lack of static type checking that's the real culprit, its the data-oriented style of programming that is. If you modeled your data with generic data-structures even in Haskell, Java, C# or any other statically typed language, you'd have the same issue.
If Clojure used abstract data-types (ADTs) like is often the case in statically typed languages, things would already be simpler.
This is how other languages work, all "entities" are created as ADTs, it has pros/cons off course, which is why Clojure tend to favour the data-oriented approach where you'd just do: But as you see, this makes it harder to know what a Name is and what's possibly available on it.altrunox | 4 years ago
Great article, love Clojure, unfortunately couldn't find any work with it when I tried, I managed to flop in the only interview I got :( Still, I miss it sometimes when I'm writing C#.
cgopalan | 4 years ago
Another good report about what Clojure does well is this article by metabase: https://medium.com/@metabase/why-we-picked-clojure-448bf759d...
I have had the pleasure of contributing to their code since we used their product at a previous company I worked at, and I must say I am sold on Clojure. Definitely a great language to have in your toolbox.
dgb23 | 4 years ago
I found one of the perceived weaknesses of Clojure (in this article), it being dynamically typed, is a tradeoff rather than a pure negative. But it applies that tradeoff differently than dynamic languages I know otherwise and that difference is qualitative: It enables a truly interactive way of development that keeps your mind in the code, while it is running. This is why people get addicted to Lisp, Smalltalk and similar languages.
> To understand a program you must become both the machine and the program.
- Epigrams in Programming, Alan Perlis
Two of the big advantages of (gradually-) typed languages are communication (documentation) and robustness. These can be gained back with clojure spec and other fantastic libraries like schema and malli. What you get here goes way beyond what a strict, static type systems gets you, such as arbitrary predicate validation, freely composable schemas, automated instrumentation and property testing. You simply do not have that in a static world. These are old ideas and I think one of the most notable ones would be Eiffel with it's Design by Contract method, where you communicate pre-/post-conditions and invariants clearly. It speaks to the power of Clojure (and Lisp in general) that those are just libraries, not external tools or compiler extensions.
hota_mazi | 4 years ago
In 2021, I find it hard to justify using a dynamically typed language for any project that exceeds a few hundreds of lines. It's not a trade off, it's a net loss.
The current crop of statically typed languages (from the oldest ones, e.g. C#, to the more recent ones, e.g. Kotlin and Rust) is basically doing everything that dynamically typed languages used to have a monopoly on, but on top of that, they offer performance, automatic refactorings (pretty much impossible to achieve on dynamically typed languages without human supervision), fantastic IDE's and debuggability, stellar package management (still a nightmare in dynamic land), etc...
sova | 4 years ago
I must respectfully disagree with the points you've brought up.
throwaway_fjmr | 4 years ago
Can you elaborate why? To be honest, I don't have experience with large-scale Clojure codebases, but I have my fair share working on fairly hefty Python and Perl projects, and I tend to think that the parent commenter is mostly right. What makes you think they are incorrect?
uDontKnowMe | 4 years ago
Not who you are responding to, but the common idea that static types are all win and no cost has become very popular these days, but isn't true, it's just that the benefits of static typing are immediately apparent and obvious, but their costs are more diffuse and less obvious. I thought this was a pretty good write up on the subject that gets at a few of the benefits https://lispcast.com/clojure-and-types/
Just to name some of the costs of static types briefly:
* they are very blunt -- they will forbid many perfectly valid programs just on the basis that you haven't fit your program into the type system's view of how to encode invariants. So in a static typing language you are always to greater or lesser extent modifying your code away from how you could have naturally expressed the functionality towards helping the compiler understand it.
* Sometimes this is not such a big change from how you'd otherwise write, but other times the challenge of writing some code could be virtually completely in the problem of how to express your invariants within the type system, and it becomes an obsession/game. I've seen this run rampant in the Scala world where the complexity of code reaches the level of satire.
* Everything you encode via static types is something that you would actually have to change your code to allow it to change. Maybe this seems obvious, but it has big implications against how coupled and fragile your code is. Consider in Scala you're parsing a document into a static type like.
//...In this example, what happens if there are new fields on Records or Tags? Our program can't "pass through" this data from one end to an other without knowing about it and updating the code to reflect these changes. What if there's a new Tag added? That's a refactor+redeploy. What if the Category tag adds a new field? refactor+redeply. In a language as open and flexible as Clojure, this information can pass through your application without issue. Clojure programs are able to be less fragile and coupled because of this.
* Using dynamic maps to represent data allows you to program generically and allows for better code reuse, again in a less coupled way than you would be able to easily achieve in static types. Consider for instance how you would do something like `(select-keys record [:id :create-ts])` in Scala. You'd have to hand-code that implementation for every kind of object you want to use it on. What about something like updating all updatable fields of an object? Again you'll have to hardcode that for all objects in scala like
all this is specific code and not reusable! In clojure, you can solve this for once and for all! I think Rich Hickey made this point really well in this funny rant https://youtu.be/aSEQfqNYNAc.Anyways I could go on but have to get back to work, cheers!
codingkoi | 4 years ago
Your third point about having to encode everything isn’t quite true. Your example is just brittle in that it doesn’t allow additional values to show up causing it to break when they do. That’s not a feature of static type systems but how you wrote the code.
This blog post[1] has a good explanation about it, if you can forgive the occasional snarkyness that the author employs.
In a dynamic system you’re still encoding the type of the data, just less explicitly than you would in a static system and without all the aid the compiler would give you to make sure you do it right.
[1]: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...
jhhh | 4 years ago
I think many peoples' experience is that most real world data models aren't as perfect as making up toy examples in blog posts. Requirements and individuals change over time. You can make an argument that in a perfect world with infinite time and money that static typing may be better because you can always model things precisely, but whether you can do that practically over longer periods of time should be a debatable question.
uDontKnowMe | 4 years ago
I've seen this article and I applaud it for addressing the issue thoroughly but I still am not convinced that static typing as we know it is as flexible and generic as dynamic typing. Let's go at this from an other angle, with a thought experiment. I hope you won't find it sarcastic or patronizing, just trying to draw an analogy here.
So, in statically typed languages, it is not idiomatic to pass around heterogeneous dynamic maps, at least in application code, like it is in Ruby/Clojure/etc. But one analogy we can draw which could drive some intuition for static typing enthusiasts is to forget about objects and consider lists. It is perfectly familiar to Scala/Java/C# programmers to pass around Lists, even though they're highly dynamic. So now think about what programming would be like if we didn't have dynamic lists, and instead whenever you wanted to build a collection, you had to go through the same rigamarole that you have to when defining a new User/Record/Tags object.
So instead of being able to use fully general `List` objects, when you want to create a list, that will be its own custom type. So instead of
you'll have to do: This represents what we're trying to do much more accurately and type-safely than with dynamic Lists, but what is the cost? We can't append to the list, we can't `.map(...)` the list, we can't take the sum of the list. Well, actually we can! So what's the problem? I've shown that the statically defined list is can handle the cases that I initially thought were missing. In fact, for any such operation you are missing from the dynamic list implementation, I can come up with a static version which will be much more type safe and more explicit on what it expects and what it returns.I think it's obvious what is missing, it's that all this code is way too specific, you can't reuse any code from List4 in List5, and just a whole host of other problems. Well, this is pretty much exactly the same kinds of problems that you run into with static typing when you're applying it to domain objects like User/Record/Car. It's just that we're very used to these limitations, so it never really occurs to us what kind of cost we're paying for the guarantees we're getting.
That's not to say dynamic typing is right and static typing is wrong, but I do think that there really are significant costs to static typing and people don't think about it.
codingkoi | 4 years ago
I’m not sure I follow your analogy. I think the dynamism of a list is separate from the type system. I can say I have a list of integers but that doesn’t limit its size.
I can think of instances where that might be useful and I think there’s even work being done in that direction in things like Idris that I really know very little about.
There are trade offs in everything. I’m definitely a fan of dynamic type systems especially things like Lisp and Smalltalk where I can interact with the running system as I go, and not having to specify types up front helps with that. Type inference will get you close to that in a more static system, but it can only do so much.
The value I see in static type systems comes from being able to rely on the tooling to help me reason about what I’m trying to build, especially as it gets larger. I think of this as being something like what Doug Englebert was pointing at when he talked about augmented intelligence.
I use Python at work and while there are tools that can do some pretty decent static analysis of it, I find myself longing for something like Rust more and more.
Another example I would point to beyond the blog post I previously mentioned is Rust’s serde library. It totally allows you to round trip data while only specifiying the parts you care about. I don’t think static type systems are as static as most like to think. It’s more about knowns and unknowns and being explicit about them.
xfer | 4 years ago
It is absolutely possible to have the same type for values that have the same shape.
You can have a `Map k v` that is a record that dynamic languages have that they call object/map.(make k/v Object or Dynamic if you want)
You don't need to create a new type with precise information if you just want that(no you don't need to instantiate type params everywhere). There is definitely limitations in type-systems (requiring advanced acrobatics) but most programs don't run into them and HM type system (https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_sy...) has stood the test of time.
For a great introduction on the idea of a type system, see: https://www.youtube.com/watch?v=brE_dyedGm0 .
throwaway_fjmr | 4 years ago
> In a language as open and flexible as Clojure, this information can pass through your application without issue. Clojure programs are able to be less fragile and coupled because of this.
Or this can wreak havoc :) Nothing stops you from writing Map<Object, Object> or Map[Any, Any], right?
uDontKnowMe | 4 years ago
That's true! But now we'll get into what is possible vs what is idiomatic, common, and supported by the language/stdlib/tooling/libraries/community. If I remember correctly, Rich Hickey did actually do some development for the US census, programming sort of in a Clojure way but in C#, before creating Clojure. But it just looked so alien and was so high-friction that he ended up just creating Clojure. As the article I linked to points out, "at some point, you're just re-implementing Clojure". That being said, it's definitely possible, I just have almost never seen anyone program like that in Java/Scala.
ud_visa | 4 years ago
Let me address your criticism from Scala's point of view
> they are very blunt
I'm more blunt than the complier usually. I really want 'clever' programs to be rejected. In rare situations when I'm sure I know something the complier doesn't, there are escape hatches like type casting or @ignoreVariace annotation.
> the problem of how to express your invariants within the type system
The decision of where to stop to encode invariants using the type system totally depends on a programmer. Experience matters here.
> Our program can't "pass through" this data from one end to an other
It's a valid point, but can be addressed by passing data as tuple (parsedData, originalData).
> What if there's a new Tag added? What if the Category tag adds a new field?
If it doesn't require changes in your code, you've modelled your domain wrong - tags should be just a Map[String, String]. If it does, you have to refactor+redeploy anyway.
> What about something like updating all updatable fields of an object
I'm not sure what exactly you meant here, but if you want to transform object in a boilerplate-free way, macroses are the answer. There is even a library for this exact purpose: https://scalalandio.github.io/chimney/! C# and Java have to resort to reflection, unfortunately.
outworlder | 4 years ago
> In 2021, I find it hard to justify using a dynamically typed language for any project that exceeds a few hundreds of lines. It's not a trade off, it's a net loss.
Only if you are skimping on tests. There's a tradeoff here - "dynamically typed" languages generally are way easier to write tests for. The expectation is that you will have plenty of them.
Given that most language's type systems are horrible (Java and C# included) I don't really think it's automatically a net gain. Haskell IS definitely a net gain, despite the friction. I'd argue that Rust is very positive too.
Performance is not dependent on the type system, it's more about language specification (some specs paint compilers into a corner) and compiler maturity. Heck, Javascript will smoke many statically typed languages and can approach even some C implementations(depending on the problem), due to the sheer amount of resources that got spent into JS VMs.
Some implementations will allow you to specify type hints which accomplish much of the same. Which is something you can do on Clojure by the way.
Automatic 'refactorings' is also something that's very language dependent. I'd argue that any Lisp-like language is way easier for machines to process than most "statically typed" languages. IDEs and debugability... have you ever used Common Lisp? I'll take a condition system over some IDE UI any day. Not to mention, there's less 'refactoring' needed.
Package management is completely unrelated to type systems.
Rust's robust package management has more to do with it being a modern implementation than with its type system. They have learned from other's mistakes.
Sure, in a _corporate_ setting, where you have little control over a project that spans hundreds of people, I think the trade-off is skewed towards the most strict implementation you can possibly think of. Not only type systems, but everything else, down to code standards (one of the reasons why I think Golang got popular).
In 2021, I would expect people to keep the distinction between languages and their implementations.
blacktriangle | 4 years ago
Here's what I've noticed with my tests and dynamic languages. I'll get type errors that static typing would have caught. However those errors occur in places I was missing testing of actual functionality. Had I had the functionality tests, then the type error would have been picked up by my tests. And had I just had static typing, the type system would not have been enough to prove the code actually works, so I would have needed tests anyways.
Point being, I don't really buy that a static type system saves me any time writing and maintaining tests, because type systems are totally unable to express algorithms. And with a working test suite (which you will need regardless of static vs dynamic) large refactors become just as mechanical in dynamic languages as they are in static languages.
tsss | 4 years ago
> type systems are totally unable to express algorithms
You don't know much about types if you think that.
As for dynamic typing "helping" you to find code that you need to write tests for: There are already far more sophisticated static analysis tools to measure code coverage.
daxfohl | 4 years ago
Yeah, I had a fairly large (about a year of solo dev work) app that I maintained both Clojure and F# ports of, doing a compare and contrast of the various language strengths. One day I refactored the F# to be async, a change that affected like half the codebase, but was completed pretty mechanically via changing the core lines, then following the red squigglies until everything compiled again, and it basically worked the first time. I then looked at doing the same to the Clojure code, poked at it a couple times, and that was pretty much the end of the Clojure port.
dharmaturtle | 4 years ago
Hey, so my my career path has been C# (many years) -> F# (couple years) -> Clojure (3 months). I understand multithreading primarily through the lens of async/await, and have been having trouble fully grokking the Clojure's multithreading. One of the commandments of async/await is don't block: https://blog.stephencleary.com/2012/07/dont-block-on-async-c...
Which is why the async monad tends to infect everything. Clojure, as far as I can tell so far, doesn't support anything similar to computation expressions. So I'm guessing your "poked at it a couple times" was something like calling `pmap` and/or blocking a future? All my multithreaded Clojure code quickly blocks the thread... and I can't tell if this is idiomatic or if there's a better way.
didibus | 4 years ago
Multi-threaded code is normally not implemented in an async style, but instead is done where each thread of execution is synchronous.
Async style comes into play generally for languages that lack real threads, or as a way to manage callbacks (even if single threaded), or in order to wait for blocking IO without the need for a real thread.
So ya, it's idiomatic to use blocking to coordinate between different threads in Clojure, same as Java.
Java decided to work on making stackful coroutines instead of stackless like C#. That requires a lot more work, but should be coming eventually to Java. At that point, your "blocking" code in Clojure will no longer block a real thread, but a lightweight fiber instead. But patience is needed for it.
In the meantime, if you're dealing with non-blocking IO that operates with callback semantics or other callback style code, what you can do in Clojure to make working with that easier is use one of:
> core.async - https://github.com/clojure/core.async
> Promesa - https://github.com/funcool/promesa
> Missionary - https://github.com/leonoel/missionary
> Missionary's lower level coroutine lib - https://github.com/leonoel/cloroutine/blob/master/doc/02-asy...
daxfohl | 4 years ago
Not even. It was opening it, looking, realizing it would take a couple weeks, and going back to F#. I did this a couple times before fully giving up.
IIRC/IIUC, Clojure's async support is closer to Go's (I've never used go), in the form of explicit channels. Though you can wrap that in a monad pretty easily, which I did for fun one day (https://gist.github.com/daxfohl/5ca4da331901596ae376). But neither option was easy to port AFAICT before giving up.
Note it's possible that porting async functionality to Clojure may have been easier that I thought at the time. Maybe adding some channels and having them do their thing could have "just worked". I was used to async requiring everything above it to be async too. But maybe channels don't require that, and you can just plop them in the low level code and it all magically works. A very brief venture into Go since then has made me wonder about that.
sooheon | 4 years ago
Sounds more like you ran into a conflict of mental model and language feature, not necessarily that the language couldn’t achieve your goal simply.
daxfohl | 4 years ago
Yeah, quite possible. I haven't worked on the project in ~six years and lost all context, but I'd revisit it and see if perhaps there was a simple solution if any of it was still current.
gnaritas | 4 years ago
Not remotely true.
[Deleted] | 4 years ago
jhgb | 4 years ago
> automatic refactorings (pretty much impossible to achieve on dynamically typed languages without human supervision)
...are we talking about the thing pioneered by Smalltalk's Refactoring Browser?
chakkepolja | 4 years ago
My question is how does that work in a dynamically typed language? In static typed language we can know scope & type of a variable and we can't change much in runtime.
bcrosby95 | 4 years ago
I find immutability way more important.
I don't pick Clojure for its dynamic typing, I pick it for other reasons. I've tried Haskell but it really doesn't seem to mesh with the way I tend to develop a program. But I would love to have more static languages with the pervasive immutability of Clojure.
JackMorgan | 4 years ago
I really like F# for this, it's like Haskell-lite
joelbluminator | 4 years ago
It's your opinion though, there's nothing scientific about what you're saying. Take mocking for example, in Ruby/Rails it's a breeze. In Java you need to invent a dependency injection framework (Spring) to do it.
de_keyboard | 4 years ago
The best response from the statically-typed world is functional programming and explicit dependencies (Haskell, OCaml, F#), which makes mocking unnecessary most of the time. OOP (Java, C#) is not the true standard for static-typing, just the most common one.
throwaway_fjmr | 4 years ago
I think you are mistaken. Mocking and DI frameworks are two unrelated concepts. There is nothing in Java that forces you to use a DI framework, e.g., Spring if you want to use mocks during testing.
joelbluminator | 4 years ago
Let's say I have a class called User and in it a method that says the current time. So User#say_current_time which simply accesses the Date class (it takes no arguments).
Can you show me how you would mock the current time of that method in Java?
It's one line of Ruby/Javascript code to do that.
lkitching | 4 years ago
If you want to use DI, in java 8 you could inject a java.time.Clock instance in the constructor and provide a fixed instance at the required time in your test e.g.
although it would be better design to have sayCurrentTime take a date parameter instead of depending on an external dependency.joelbluminator | 4 years ago
Yes that was my point. You don't need DI or to structure your code any differently in Ruby/JS/Python. You just mock a method.
lkitching | 4 years ago
In my experience the need to mock out individual methods like this is an indication that the code is badly structured in the first place. The time source is effectively a global variable so in this example you'd want to pass the time as a parameter to `sayCurrentTime` and avoid the need to mock anything in the first place. A lot of C#/java codebases do seem to make excessive use of mocks and DI in this way though.
Jach | 4 years ago
Without using a mock framework, assuming User#say_current_time isn't a private or static method then:
If it is private and/or static, you can get around it without having to change the code, but if you own the code, you should just do that... Often the change will be as simple as replacing some method's raw usage of Date.now() with a local say_curent_time() method that uses it or some injected dependency just so you can mock Date.now() without hassle.But your point further down that in Java you have to think about your code structure more to accommodate tests is valid. I think it's easy to drink the kool-aid and start believing that many code structuring styles that enable easier testing in Java are actually very often just better styles regardless of language, but you're not going to really see the point if you do nothing but Ruby/JS where you can get away with not doing such things for longer. Mostly it has to do with dynamic languages offering looser and later and dynamic binding than static languages (which also frequently makes them easier to refactor even if you don't have automated tools). One big exception is if your language supports multiple dispatch, a lot of super ugly Java-isms go away and you shouldn't emulate them. The book Working Effectively with Legacy Code is a good reference for what works well in Java and C++ (and similar situations in other languages), it's mostly about techniques for breaking dependencies.
throwaway_fjmr | 4 years ago
I am assuming this is easier in Ruby because you can monkey patch classes?
Mockito in Java has a nifty way of doing this with Mockito.mockStatic:
Or you can pass a Clock instance and use .now(clock). That Clock then can be either a system clock or a fixed value.joelbluminator | 4 years ago
> I am assuming this is easier in Ruby because you can monkey patch classes?
Yes, that was my point. I see it's possible in Java though, hurts my eyes a bit but possible :)
mypalmike | 4 years ago
I'll take clean contractual interfaces (aka actual principle of least surprise) over "I can globally change what time means with one line of code!" on large projects every time.
hota_mazi | 4 years ago
joelbluminator | 4 years ago
OK. first I could be ignorant about Java since I haven't touched it in more than a decade. Which library is doing that? And also what is mock(User.java) returning - is it an actual User instance or a stub? I want a real User instance (nothing mocked in it) with just the one method mocked.
And again if this is possible I will admit ignorance and tip my hat at the Java guys.
throwaway_fjmr | 4 years ago
I think what you want is a "spy" (partial mock), not a full "mock", but yes, both are possible. You can partially mock classes, i.e., specific methods only. Syntax is almost the same, instead of mock(User.class) you write spy(User.class).
hota_mazi | 4 years ago
It's Mockito [1], which has been a standard for a while. There are other libraries and they use different strategies to provide this kind of functionalities (dynamic proxies, bytecode weaving, annotation processing, etc...).
[1] https://site.mockito.org/
joelbluminator | 4 years ago
And ... is the whole user being mocked or just the method?
vincnetas | 4 years ago
It creates a stub, but you can also configure it to pass any method calls to original implementation. You should be tiping your hat i think.
https://javadoc.io/static/org.mockito/mockito-core/3.11.2/or...
User mock = mock(User.java)
Should have been
User mock = mock(User.class)
hota_mazi | 4 years ago
Ah oops, I've been writing exclusively Kotlin for several years, my Java is becoming rusty (no pun intended).
mmcdermott | 4 years ago
In theory, I agree, but I don't think that holds terribly true in practice.
One of the ideas behind IoC frameworks (which build on top of DI) is that you could swap out implementation classes. For a great deal of software (and especially in cloud-hosted, SaaS style microservice architecture) the test stubs are the only other implementations that ever get injected.
Most code bases could ditch IoC if Java provided a language-level construct, even if that construct were only for the test harness.
Jach | 4 years ago
Java has a mechanism, just pass alternate implemenations in constructors. If you must, a setter method. For most code you don't need to bring in the overhead of Spring, and @Autowired isn't really more convenient typing wise. Plus your unit tests become trivial, they're just POJOs with @Test annotations.
Spring is great when you need that dynamic control at runtime (especially when code dependencies are separated by modules) but you're just aping what good dynamic languages like Clojure or Common Lisp give you for free. But I can't complain too much, developing modern Java with its popular frameworks and with JRebel is getting closer to the Lisp experience every year, I'd rather have that than for Java to remain stagnate like in its 1.6/1.7 days.
hota_mazi | 4 years ago
The fact that there are such libraries in existence means that there is no pain associated to this particular activity. Not only do you get great mocking frameworks, they are actually very robust and benefit from static types.
Mocking dynamically typed languages is monkey patching, something that the industry has been moving away from for more than a decade. And for good reasons.
joelbluminator | 4 years ago
> The fact that there are such libraries in existence means that there is no pain associated to this particular activity
I can say the same about Rails + RSpec. It exists therefore it's good.
> Mocking dynamically typed languages is monkey patching, something that the industry has been moving away
That's a reach. There are millions of javascript/python/php/ruby/elixir devs that don't use types or annotations. They mock. "The industry" isn't one cohesive thing.
[Deleted] | 4 years ago
brundolf | 4 years ago
Maybe I've just never given it a chance, but I've never understood the appeal of being able to modify code in-memory while it's running.
I like a REPL for testing things out, or for doing quick one-off computations, but that's it. I would never want to, say, redefine a function in memory "while the code is running". Not just because of ergonomics, but because if I decide to keep that change, I now have to track down the code I typed in and manually copy it back over into my source files (assuming I can still find it at all). And if I make a series of changes over a session, the environment potentially gets more and more diverged from what's in sourced if I forget to copy any changes over. So I'd often want to re-load from scratch anyway, at least before I commit.
Am I missing something? Am I misunderstanding what people mean when they talk about coding from a REPL?
Jach | 4 years ago
You can get an approximate taste of what it's like in plain old Java + JRebel, it's seriously about the same as trying to do it with Python + something like Flask. Start up a big application server in debug mode with Eclipse/IntelliJ, be annoyed that it takes 2+ minutes to start after everything's compiled. Now you want to work on a story, or some bug. You can make changes, save, it recompiles just that file (incremental compilation is a godsend in itself), and hotswaps it into the running application memory. No need to restart anything (with JRebel, for most common kinds of changes; without JRebel, only for some changes). It's particularly useful when you're debugging, you find the problem, change the code, and re-execute immediately to verify to yourself it's fixed.
You also can get an experience like PHP, where you just have to change and save some files, and your subsequent requests will use the new code. This is so much better than shutting down everything and restarting and is a large part of why CGI workflows dominated the web.
Common Lisp takes these experiences and dials them to 11, the whole language is built to reinforce the development style of dynamic changes, rather than an after-thought that requires a huge IDE+proprietary java agent. It's still best to use some sort of editor or IDE, and then you don't have any worry about source de-syncs -- frequently you'll make multiple changes across multiple files and then just reload the whole module and any files that changed with one function call, which you might bind to an editor shortcut, but crucially like debugging is not centrally a feature of the editor but the language; the language's plain REPL by itself is just a lot more supportive of interactive development than Python/JS/Ruby's. Clojure, and I personally think even Java with the appropriate tools, are between Python and CL for niceness of interactive development, but Clojure tends to be better than the Java IDE experience because of its other focus on immutability.
NightMKoder | 4 years ago
As others have mentioned - you’re probably talking about something like a python or node repl. Lisp repl development is not like python or node - you _work_ in the repl. The closer comparison might be between bash+node as a repl - up+enter and the like to rerun tests has an equivalent in clojure+IntelliJ. There’s no copy pasting but there are different key bindings.
One of the best parts about lisp style repl development is that you end up doing TDD automatically. You just redefine a function until it does what you want from sample data you pass in - without changing files or remembering how your test framework works. You can save the output of some http call in a top level variable and iterate on the code to process it into something useful. The code you evaluate lives in the file that will eventually house it anyway so it’s pretty common to just eval the entire file instead of just one function.
Since you don’t ever shut the repl down, developing huge apps is also quite pleasant. You only reload the code that you’re changing - not the rest of the infra so things like “memoize” can work in local development. That’s why it’s a bit closer to your bash shell in other languages.
If you’ve never tried it, I highly recommend trying the Clojure reloaded workflow [1] to build a web app with a db connection. You can really get into a flow building stuff instead of waiting for your migrations to run on every test boot.
[1] https://cognitect.com/blog/2013/06/04/clojure-workflow-reloa...
shaunxcode | 4 years ago
Yes, what you are missing is that usually you are using an editor like emacs where you can modify a specific form and send it to the repl. That way there is no chasing back through repl history for a form to paste back into a file.
sooheon | 4 years ago
Instead of typing things into a repl and copying back into source, imagine stepping through your source in a debugger.
geokon | 4 years ago
I've admittedly not played with spec, but can't you solve documenting interfaces by defining `defrecord`s ? You rarely really care about the actual types involved. You just want to know which fields you either need to provide or will recieve
roenxi | 4 years ago
Spec will give you stronger feedback than a docstring or function signature. It can tell you (in code terms, with a testable predicate) if a call to an interface wouldn't make sense.
Eg, spec can warn you when an argument doesn't make sense relative to the value of a second argument. Eg, with something like (modify-inventory {:shoes 2} :shoes -3) spec could pick up that you are about to subtract 3 from 2 and have negative shoes (impossible!) well before the function is called - so you can test elsewhere in the code using spec without having to call modify-inventory or implement specialist checking methods. And a library author can pass that information up the chain without clear English documentation and using only standard parts of the language.
You can't do that with defrecord, but it is effectively a form of documentation about how the arguments interact.
modernerd | 4 years ago
Does the spec logic typically live inside the modify-inventory function, or elsewhere? If elsewhere, what triggers it before the function is called?
teataster | 4 years ago
There is very little spec logic. It looks a lot like type declarations in typed languages.
It's usually outside the scope of functions, since you are likely going to want to reuse those declarations. For example, you can use spec to generate test cases for something like quick-check.
You can add pre and post conditions to clojure function's metadata that test wether the spec complies with the function's input/output.
k__ | 4 years ago
I think, the big issue with dynamic typing in popular languages like PHP and JavaScript are the automatic conversions.
robertlagrant | 4 years ago
This is a consequence of weak typing rather than dynamic typing. I appreciate that these are not precise terms, but being able to change something's type (dynamic) is different to the language just doing strange things when you combine types (weak).
dgb23 | 4 years ago
You mean implicit type conversions? That's a thing you can get somewhat used to. But it throws off beginners and can introduce super weird bugs, because they hide bugs in weird ways, even if you are more experienced. Yes, I find strong typing strictly better than weak typing.
An even better example of this would be Excel, the horror stories are almost incredible.
So even if your environment is dynamic, you want clarity when you made a mistake. Handling errors gracefully and hiding them are very different things. The optimal in a dynamic world is to facilitate reasoning while not restricting expression.
dangerbird2 | 4 years ago
It's always worth reminding folks that weak typing and implicit conversions can plague statically typed languages. C's implicit pointer array-to-pointer and pointer-type conversions are a major source of bugs for beginner and experienced programmers alike.
elwell | 4 years ago
Which is less of a concern considering Clojure's focus on immutability.
k__ | 4 years ago
That's what I meant.
As far as I know, some dynamic languages like Python don't have that issue.
vnorilo | 4 years ago
Agreed. I feel Lisps and SmallTalk are dynamic done right. I think the other language features that you use also influence the value from dynamic or static types. For OOP style, static types are a huge asset for refactoring and laying our architecture. On the other hand, immutable data and stateless functions (as idiomatic in clojure) make them less necessary, and also work great together with interactive development.
pjmlp | 4 years ago
Not only that, Smalltalk and Lisps are languages designed with developer experience as part of the language.
You just don't get an interpreter/compiler and have to sort everything else by yourself, no, there is a full stack experience and development environment.
lakecresva | 4 years ago
>You simply do not have that in a static world.
I keep seeing lisp people bandy about all of this design by contract/arbitrary predicate validation stuff. Can you give an example of an instance in which static types + runtime checks don't completely subsume this?
My intuition is that almost all of these methods people are talking about would have to be enforced at run-time, in which case I don't see how it's providing anything fundamentally more than writing an assertion or a conditional.
c-cube | 4 years ago
Except, of course, that specs are only tested correct, not proven correct like types would be. Types (in a reasonable static type system, not, say, C) are never wrong. In addition, specs do not compose, do they ? If you call a function g in a function f, there is no automatic check that their specs align.
undershirt | 4 years ago
Yeah, and I think this is obvious, but it certainly depends on the origin of the data being checked. We can prove the structure of “allowed” data ahead of time if we want guarantees on what’s possible inside our program. We also want a facility to check data encountered by the running program (i.e. from the user or another program.) which of course we can’t know ahead of time.
It is a design decision to be able to build a clojure system interactively while it is running, so a runtime type checker is a way for the developer to give up the safety of type constraints for this purpose—by using the same facility we already need in the real world, a way to check the structure of data we can’t anticipate.
travisjungroth | 4 years ago
> Types (in a reasonable static type system, not, say, C) are never wrong.
Oh man. This is the fundamental disagreement. Sure, you can have a type system that is never wrong in its own little world. But, that's not the problem. A lot of us are making a living mapping real world problems into software solutions. If that mapping is messed up (and it always is to some degree) then the formal correctness of the type system doesn't matter at all. It's like you got the wrong answer really, really right.
lolinder | 4 years ago
> A lot of us are making a living mapping real world problems into software solutions. If that mapping is messed up (and it always is to some degree) then the formal correctness of the type system doesn't matter at all.
If I'm understanding you correctly, you're saying statically typed language can't protect against design flaws, only implementation flaws. But implementation flaws are common, and statically typed languages do help to avoid those.
c-cube | 4 years ago
I'm not saying types always model your problem properly! That's not even well specified. I'm saying that "x has type foo" is never wrong if the program typechecks properly. That's totally different, and it means that you can rely on type annotations as being correct, up-to-date documentation. You can also trust that functions are never applied to the wrong number of arguments, or the wrong types; my point is that this guarantees more, in a less expressive way, than specs.
travisjungroth | 4 years ago
You can statically analyze specs and check them at runtime if you want.
cle | 4 years ago
> Except, of course, that specs are only tested correct, not proven correct like types would be.
Yes this is the fundamental tradeoff. Specs et al are undoubtedly more flexible and expressive than static type systems, at the expense of some configurable error tolerance. I don't think one approach is generally better than the other, it's a question of tradeoffs between constraint complexity and confidence bounds.
dgb23 | 4 years ago
Yes, I think that is one of the big weaknesses of it. You can write specs that make no sense and it will just let you. So far there is also no way to automatically check whether you are strengthening a guarantee or weaken your assumptions relative to a previous spec. In a perfect world we would have this in my opinion.
travv0 | 4 years ago
The very first property testing library was written in Haskell, as far as I know.
blacktriangle | 4 years ago
I feel like there's a missing axis in the static/dynamic debate: the language's information model.
In an OOP language, types are hugely important, because the types let you know the object's ad-hoc API. OOP types are incredibly complicated.
In lisps, and Clojure in particular, your information model is scalars, lists, and maps. These are fully generic structures whose API is the standard Clojure lib. This means that its both far easier to keep the flow of data through your program in your head.
This gives you a 2x2 matrix to sort languages into, static vs dynamic, and OOP vs value based.
* OOP x static works thanks to awesome IDE tooling enabled by static typing
* value x static works due to powerful type systems
* value x dynamic works due to powerful generic APIs
* OOP x dynamic is a dumpster fire of trying to figure out what object you're dealing with at any given time (looking right at you Python and Ruby)
scotty79 | 4 years ago
> ... such as arbitrary predicate validation, freely composable schemas, automated instrumentation and property testing ...
Why static typing makes those things impossible?
dgb23 | 4 years ago
They don't make these impossible, they typically just don't let you express these within the type system and they typically don't let you not specify your types.
I should have made clear that I'm emphasizing the advantages of being dynamic to describe and check the shape of your data to the degree of your choosing. Static typing is very powerful and useful, but writing dynamic code interactively is not just "woopdiedoo" is kind of the point I wanted to make without being overzealous/ignorant.
scotty79 | 4 years ago
I think TypeScript is best of both worlds.
Typesystem strong enough to express dynamic language and completely optional wherever you want.
thinkharderdev | 4 years ago
That largely depends on the type system. Languages like Haskell and Scala which have much more powerful type systems than C/Java/Go/etc absolutely do allow you to do those sorts of things. It is a bit harder to wrap your head around to be sure and there are some rough edges, but once you get the hang of it you can get the benefits of static typing with the flexibility of dynamic typing. See https://github.com/milessabin/shapeless or a project that I've been working on a lot lately https://github.com/zio/zio-schema.
dharmaturtle | 4 years ago
> What you get here goes way beyond what a strict, static type systems gets you, such as arbitrary predicate validation,
Is this refinement types, which most static languages provide? https://en.wikipedia.org/wiki/Refinement_type
> freely composable schemas,
My understanding is that you can compose types (and objects) https://en.wikipedia.org/wiki/Object_composition
I'm assuming that types are isomorphic with schemas for the purposes of this discussion.
> automated instrumentation
I know that C# and F# support automated instrumentation/middleware.
> and property testing. You simply do not have that in a static world.
QuickCheck has entered the chat: https://en.wikipedia.org/wiki/QuickCheck
CraigJPerry | 4 years ago
>> Is this refinement types
Well it does include that kind of behaviour but it's quite a bit more than just that. E.g. you could express something like "the parameter must be a date within the next 5 business days" - there's no static restriction. I'm not necesarily saying you should but just to give an illustrative example that there's less restrictions on your freedom to express what you need than in a static system.
>> types are isomorphic with schemas
I don't think that's a good way to think of this, you're imagining a rigid 1:1 tie of data and spec yet i could swap out your spec for my spec so that would be 1:n but those specs may make sense to compose in other data use cases so really it's m:n rather than 1:1
dharmaturtle | 4 years ago
> E.g. you could express something like "the parameter must be a date within the next 5 business days" - there's no static restriction
Hm, I don't follow. If I were to write this in F#, there would be a type `Within5BusinessDays` with a private constructor that exposes one function/method `tryCreate` which returns a discriminated union: either an `Ok` of the `Within5BusinessDays` type, or an `Error` type with some error message. Once I have the type, I can then compose it with whatever and send it wherever and since F# records are immutable, I won't have to worry about invariants not holding. And since it's a type, I have the compiler/type system on my side to help with correctness.
(Side note, this is a bad example since the type can become invalid after literally 1 second... but since Clojure has the same problem I'm just running with it.)
I'm still learning Clojure (only a few months into it), but if I were to to write a spec, I'd have to specify what to do do if the spec failed to conform - same as returning the `Error` case in F#.
> i could swap out your spec for my spec so that would be 1:n but those specs may make sense to compose in other data use cases so really it's m:n rather than 1:1
Sorry, but I'm still not following - I believe you can do the same with types, especially if the type system support generics.
CraigJPerry | 4 years ago
> If I were to write this in F#, there would be a type `Within5BusinessDays`
That’s not really the same thing - it’sa valid alternative approach but you’ve lost the benefits of a simple date - from (de)serialisation to the rich support for simple date types in libraries and other functions, the simple at-a-glance understanding that future readers could enjoy. Now the concept of date has been complected with some other niche concern.
> the type can become invalid after literally 1 second
Every system I’ve ever seen that has the concept of a business date strictly doesn’t derive it from wall clock date. E.g. it’s common that business date would be rolled at some convenient time (and most often not midnight) so you’d be free to ensure no impacts possible from the date roll.
>> I believe you can do the same with types, especially if the type system support generics
You can do something similar but you’ll need to change the system’s code.
It would be almost like gradual typing, except you could further choose to turn it off or to substitute your own types / schema without making changes to the system / code.
It’s quite a lot more flexible.
(Apols for slow reply - i 1up’d your reply earlier when i saw it but couldn’t reply then)
dharmaturtle | 4 years ago
> but you’ve lost the benefits of a simple date
I see what you mean - thanks!
> you could further choose to turn it off or to substitute your own types / schema without making changes to the system / code
This is still unclear to me. How can you make changes (turning off gradual typing/substituting your own schema) without making changes to code?
dgb23 | 4 years ago
Right! I made the dumb, typical error to write: "You simply do not have that in a static world." When I should have written: "This type of expressiveness is not available in mainstream statically typed languages".
With "freely composable" I mean that you can program with these schemas as they are just data structures and you only specify the things you want to specify. Both advantage and the disadvantage is that this is dynamic.
dharmaturtle | 4 years ago
Ah, well if you're going to shit on Go/Java/C#/C++ I won't stop you :)
scotty79 | 4 years ago
I tried to use Clojure but what put me of was that simple mistakes like missing argument or wrongly closed bracket didn't alert me until I tried running the program and then gave me just some java stack spat out by jvm running Clojure compiler on my program.
It didn't feel like a first class experience.
capableweb | 4 years ago
> didn't alert me until I tried running the program
That's because that's not how Clojure developers normally work. You don't do changes and then "run the program". You start your REPL and send expressions from your editor to the REPL after you've made a change you're not sure about. So you'd discover the missing argument when you call the function, directly after writing it.
scotty79 | 4 years ago
Interesting. How exactly that looks? Do you have files opened in your editor, change them then go into previously opened repl, and just call the functions and the new version of those function runs?
finalfantasia | 4 years ago
Thanks to the dynamic nature of Clojure programs, experienced Clojure developers use the REPL-driven development workflow as demonstrated in this video [1].
[1] https://youtu.be/gIoadGfm5T8
scotty79 | 4 years ago
From what I understand, instead of writing the file and running the file you write separate statements in the file and evaluate each of them in the repl (like with "Do it" in Smalltalk).
So what you get, after running the file afterwards from clean state might be different than the result of your selective separate manual evaluations.
This looks like exactly the opposite of the F5 workflow in the browser where you can run your program from clean state with single keypress.
I haven't watched the video till the end though maybe there's a single key that restarts the repl and runs the files from clean state here too.
At first glance you could have the same workflow with JS, but there's not much need for it because JS VMs restart very quickly and also you'd need to code in JS in very particular style, avoiding passing function and class "pointers" around and avoid keeping them in variables. I guess clojure just doesn't do that very often and just refers to functions through their global identifiers, and if that's not enough, even through symbols (like passing the #'app in this video instead of just app).
uDontKnowMe | 4 years ago
That's right. You typically would have your text editor/ide open, and the process you're developing would expose a repl port which your editor can connect to. As you edit the source code, that will automatically update the code running in the process you're debugging. See this demo of developing a ClojureScript React Native mobile app published yesterday: https://youtu.be/3HxVMGaiZbc?t=1724
[Deleted] | 4 years ago
pron | 4 years ago
> We tried VisualVM but since Clojure memory consists mostly of primitives (Strings, Integers etc) it was very hard to understand which data of the application is being accumulated and why.
You should try deeper profiling tools like JFR+JMC (http://jdk.java.net/jmc/8/) and MAT (https://www.eclipse.org/mat/).
gavinray | 4 years ago
I was going to suggest this -- inside of VisualVM, you can right-click a process and then press "Start JFR"
Then wait a bit, right click it again, and select "Dump JFR"
What you get is a Flight Record dump that contains profiling information you can view that's more comprehensive than any language I've ever seen.
I used this for the first time the other day and felt like my life has been changed.
Specifically, if you want to see where the application is spending it's time and in what callstacks, you can use the CPU profiling and expand the threads -- they contain callstacks with timing
There's some screenshots in an issue I filed here showing this if anyone if curious what it looks like:
https://github.com/redhat-developer/vscode-java/issues/2049
Thanks Oracle.
user3939382 | 4 years ago
Walmart Labs was a step in this direction.. but we need some big companies to standardize around Clojure to jumpstart the ecosystem of knowledge, libraries, talent, etc. I’ve spoken to engineering hiring managers at fairly big companies and they’re not willing to shift to a niche language based only on technical merits but without a strong ecosystem.
If we don’t get some big companies to take on this roll the language is going nowhere.
I’m saying this because I’m a huge fan of Clojure (as a syntax and language, not crazy about the runtime characteristics) and I hope I get the opportunity to use it.
iLemming | 4 years ago
> If we don’t get some big companies to take on this roll
- Cisco - has built their entire integrated security platform on Clojure
- Walmart Labs and Sam's club - have some big projects in Clojure
- Apple - something related to the payment system
- Netflix and Amazon, afaik they use Clojure as well
even NASA uses Clojure.
I think the language "is going somewhere"...
user3939382 | 4 years ago
role* lol we made the same mistake. There is some adoption to be sure. But look at Google Trends for clojure.
iLemming | 4 years ago
What does Google Trends have to do with a programming language? PHP is trending, and Clojure is not, perhaps because Clojure gives you a lot fewer reasons to google stuff up?
I myself rarely use Google to find a solution to a problem, and certainly almost never have to google shit like: "how to open a file in Clojure"...
ivanech | 4 years ago
I started working professionally with Clojure earlier this year and this article rings true. I think the article leaves out a fourth downside to running on the JVM: cryptic stack traces. Clojure will often throw Java errors when you do something wrong in Clojure. It's a bit of a pain to reason about what part of your Clojure code this Java error relates to, especially when just starting out.
alaq | 4 years ago
How did you make the switch? Were you already working for the same company? Did you already know Clojure, from open source, or side projects?
ivanech | 4 years ago
I work at Ladder [0], and almost everything is done in Clojure/ClojureScript here. I had no previous experience in Clojure – Ladder ramps you if you haven't used it before. My interview was in Python. We're currently hiring senior engineers, no Clojure experience necessary [1].
[0] https://www.ladderlife.com/
[1] https://boards.greenhouse.io/ladder33/jobs/2436386
finalfantasia | 4 years ago
To be fair, this is not unique to Clojure. You need to deal with stack traces no matter what as long as you're using any programming language that targets the JVM (even statically type-checked languages like Scala). There are some great articles [1][2] that discuss various simple techniques helpful for debugging and dealing with stack traces.
[1] https://eli.thegreenplace.net/2017/notes-on-debugging-clojur...
[2] https://cognitect.com/blog/2017/6/5/repl-debugging-no-stackt...
rockostrich | 4 years ago
I've never really had a problem with stack traces in Scala. Every once in a while you hit a cryptic one that's buried in Java library code, but for the most part they're runtime errors that are due to incompletely tested code or some kind of handled error with a very specific message.
aliswe | 4 years ago
> ... and the question regarding choosing Clojure as our main programming language rose over and over again
If I find myself having to repeat myself justifying a certain decision time and time again, it's an indicator that the decision needs to be revised to be something which is a more intuitive fit for the organization.
outworlder | 4 years ago
That's not a good indication that the decision was or was not correct. Only that it currently runs against whatever the established practice is. Sometimes "the way things have always been done" is just wrong.
This is unlikely to be the case in the choice of programming languages. Some may be a bad fit, some may have ecosystems that are unpleasant to use, but it's generally not the biggest problem an organization will have.
lostcolony | 4 years ago
Not really; it's like stoplights. You're going to be interrupted and therefore notice the red lights, and just sail easily through and thus not notice the green lights. Likewise, you're going to notice the pain points, but need to take a minute to reflect to notice the benefits.
Really, if repeating the same justifications convinces people, then the problem isn't the justifications.
joelbluminator | 4 years ago
I donno why you're being downvoted, it's a questionable decision and probably the company would have been better off with Python/PHP/Node. Hiring and onboarding are extremely important for a startup. You know what else? Finding answers to common questions on Google/Stackoverflow; I am now working with Ember and can tell you guys you take a 50% productivity hit by using a tool that's obscure on Google. Sure once you become super familiar with a tool that matters less, but that takes time. Much more time. React/Angular may be an inferior tool to Ember but the fact that you can get answers to almost any question is priceless. The community size is super important. The frameworks are super important (is there a Closure equivalent to Rails/Django/Laravel in community size, in battle testedness? I really doubt it).
That being said, I salute these brave companies for sticking to these obscure languages. Do we want to live in a world where there's only 3 languages to do everything? Even 10 sounds boring. Hell, even a fantastic tool like Ruby is considered Niche in certain parts of the world. I don't want a world without Ruby so I don't want a world without Closure.
aliswe | 4 years ago
> I donno why you're being downvoted
Oh that's easy - the voting system is a way to know how conformant you are to other opinions coffee smile
iLemming | 4 years ago
> Hiring and onboarding are extremely important for a startup.
If you're a small company, you usually cannot afford to hire "mediocre" talent. It is much more expensive to undo the crapola they'd implement. Trying to hire those who are at least interested in learning and using languages like Clojure, Rust, Haskell, Elixir, Elm, etc., is a very good quality filter. ROI from hiring a smaller number of Clojure devs, rather than a few more "regular" engineers - is much higher.
> Finding answers to common questions on Google/Stackoverflow;
Clojure gives you far fewer reasons for Googling things than other language ecosystems. It is dense language and inspires you to write smaller functions, decreasing the surface area for the problem. Most of the time, asking questions in Clojurians Slack sends you halfway through the solution.
> I salute these brave companies for sticking to these obscure languages
They do not choose Clojure for the shtick; Clojure is a highly pragmatic and immensely productive instrument. There are many "success stories" with small and medium-sized companies. A few large companies like Cisco, Apple, Walmart, et al., actively develop in Clojure.
The same can be said about the engineers. They don't choose Clojure because "they hate Java". You can check any Clojure surveys of the past. Most Clojure engineers are experienced and "tired" developers. Seasoned hackers who have seen the action. For most of them - Clojure is a deliberate choice. Many of them landed in it after trying various other alternatives.
yakshaving_jgt | 4 years ago
Alternatively, you could document the thought process that lead up to the decision and you can point the unenlightened to the documentation instead of having to repeat yourself.