Laya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)

111 points by putna 6 hours ago on hackernews | 21 comments

Nice! I just wanted to see what Laya does and put it behind Cloudflare's typesafe/jev API:

curl https://laya.inference.zaitlabs.com/ai/run \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Help! My payouts have been failing for 3 days.",
    "questions": {
      "is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"}
    }
  }'
{"answers":{"is_urgent":{"type":"noul","noul":0.7894}}}

https://laya.inference.zaitlabs.com/terms