I’ve been looking around for a scripting language that:
- has a cli interpreter
- is a “general purpose” language (yes, awk is touring complete but no way I’m using that except for manipulating text)
- allows to write in a functional style (ie. it has functions like map, fold, etc and allows to pass functions around as arguments)
- has a small disk footprint
- has decent documentation (doesn’t need to be great: I can figure out most things, but I don’t want to have to look at the interpter source code to do so)
- has a simple/straightforward setup (ideally, it should be a single executable that I can just copy to a remote system, use to run a script and then delete)
Do you know of something that would fit the bill?
Here’s a use case (the one I run into today, but this is a recurring thing for me).
For my homelab I need (well, want) to generate a luhn mod n check digit (it’s for my provisioning scripts to generate synchting device ids from their certificates).
I couldn’t find ready-made utilities for this and I might actually need might a variation of the “official” algorithm (IIUC syncthing had a bug in their initial implementation and decided to run with it).
I don’t have python (or even bash) available in all my systems, and so my goto language for script is usually sh (yes, posix sh), which in all honestly is quite frustrating for manipulating data.
i use mostly python and lua.
Lua
Shutters
Why aren’t python and bash be available in all your systems? Which languages would be?
I would’ve recommended python, otherwise perl or Haskell (maybe Haskell’s too big) or something, but now I’m worried that whatever reason makes python undoable also makes perl etc. undoable
Why aren’t python and bash be available in all your systems?
Among others, I run stuff on alpine and openwrt.
I don’t need to run these scripts everywhere (strictly speaking, I don’t need the homlab at all), but I was wondering if there’s something that I can adopt as a default goto solution without having to worry about how each system is packaged/configured.
As for python, I doubt the full version would fit in my router plus as said I don’t want to deal with libraries/virtualenvs/… and (in the future) with which distro comes with python3 vs pyton4 (2 vs 3 was enough). Openwrt does have smaller python packages, but then I would be using different implementations on different systems: again something I’d rather not deal with.
As for perl, it would be small enough, but I find it a bit archaic/esoteric (prejudice, I know), plus again I don’t want to deal with how every distro decides to package the different things (eg. openwrt has some 40+ packages for perl - if I were doing serious development that would be ok, but I don’t want to worry about that for just some scripts).
Is compiling scripts an option? Aka compiling them in C, C++, Rust, whatever for your router on another machine, and copying and executing those binaries on your router?
if there’s something that I can adopt as a default goto solution without having to worry about how each system is packaged/configured.
Go is probably your best bet. Simple to use, and you can compile it so it runs everywhere
I found installing Go-sdk a total PiTA. It is okay as a developer environment. But bash + gnu utils + core utils seem much more sane to me.
Of course I mostly work with Linux systems and hardly ever have to deal with Scripting for Windoze.
Sounds like you want MicroPython. It’s definitely available on OpenWrt and AlpineLinux and has a very small footprint.
If you don’t like Python, have a look at Lua/luajit.
You’ve defined yourself into an impossible bind: you want something extremely portable, universal but with a small disk imprint, and you want it to be general purpose and versatile.
The problem is that to be universal and general purpose, you need a lot of libraries to interact with whatever type of systems you might have it on (and the peculiarities of each), and you need libraries that do whatever type of interactions with those systems that you specify.
E.g. under-the-hood, python’s
open("<filename>", 'r')is a systemcall to the kernel. But is that Linux? BSD? Windows NT? Android? Mach?What if you want your script to run a CLI command in a subshell? Should it call “cmd”? or “sh”? or “powershell”? Okay, okay, now all you need it to do is show the contents of a file… But is the command “cat” or “type” or “Get-FileContents”?
Or maybe you want to do more than simple read/write to files and string operations. Want to have graphics? That’s a library. Want serialization for data? That’s a library. Want to read from spreadsheets? That’s a library. Want to parse XML? That’s a library.
So you’re looking at a single binary that’s several GBs in size, either as a standalone or a self-extracting installer.
Okay, maybe you’ll only ever need a small subset of libraries (basic arithmetic, string manipulation, and file ops, all on standard glibc gnu systems ofc), so it’s not really “general purpose” anymore. So you find one that’s small, but it doesn’t completely fit your use case (for example, it can’t parse uci config files); you find another that does what you need it to, but also way too much and has a huge footprint; you find that perfect medium and it has a small, niche userbase… so the documentation is meager and it’s not easy to learn.
At this point you realize that any language that’s both easy to learn and powerful enough to manage all instances of some vague notion of “computer” will necessarily evolve to being general purpose. And being general purpose requires dependencies. And dependencies reduce portability.
At this point your options are: make your own language and interpreter that does exactly what you want and nothing more (so all the dependencies can be compiled in), or decide which criteria you are willing to compromise on.
Mate, I came on here to post Haskell as a semi-ironic ‘joke’ and it’s included in the top comment. You’ve made my day.
I mean, it’s a functional programming language and an incredibly good one. But it probably has a far too big footprint for their use-case…
Oh, I know. I adore Haskell.
guile scheme or Julia
Neither of them seem to be a single file, and both seem to have several dependencies, at least that’s the case with the Homebrew versions.
@gomp I like TypeScript.
I used Python for 15 years or so until they changed from v2 to v3. At that point I realised I couldn’t understand my old code because it lacked types, so I got discouraged with that. So rather than learn v3 I stopped using it.
Perl is a disaster. sh is good for shell scripts but let’s not stretch it.
TypeScript can use all the JS libraries and runs on node which is supported by all sorts of platforms. Yes there are a few holes in the type safety, so don’t do that.
The internet is full of “how to do X in JS”. You can read them and add the types you need.
I can’t really think of anything that’s less frustrating than sh and ticks all your boxes. You can try TCL but it’s bound to be a shit show. It was painful to use two decades ago.
Perl is a step up in terms of developer comfort, but it’s at the same time too big and too awkward to use.
Maybe a statically linked Python?
Perl is a step up in terms of developer comfort, but it’s at the same time too big and too awkward to use.
How do you mean?
It’s already on nearly every distro, so there’s no core size unless you lean into modules. The scripts aren’t exactly big either.
He doesn’t have bash. I’m not sure I’ve seen a system this millennium with Perl but not bash.
Try it now - type perl. It’s a dependency on a huge amount of core system tools.
OP is on OpenWRT (a router distro), and Alpine. Those distros don’t come with very much by default, and perl is not a core dependency for any of their default tools. Neither is python.
Based on the way the cosmo project has statically linked builds of python, but not perl, I’m guessing it’s more difficult to create a statically linked perl. This means that it’s more difficult to put perl on a system where it isn’t already there, and that system doesn’t have a package manager*, than python or other options.
*or the the user doesn’t want to use a package manager. OP said they just want to copy a binary around. Can you do that with perl?
I was thinking about recommending TCL as a joke. My favorite thing about it is it’s “whimsicly typed.”
Micropython.
That does seem to be just one, maybe two small files, and no dependencies. And a built in map() function.
Maybe something like Elvish or Nushell could be worth a look. They have a lot of similarities to classic shells like bash, but an improved syntax and more powerful features. Basically something in between bash and Python. Not sure about disk footprint or general availability/portability though
What about Lua/Luajit?
In most scripting languages you have the interpreter binary and the (standard) libraries as separate files. But creating self-extracting executables, that clean up after themselves can easily be done by wrapping them in a shell script.
IMO, if low dependencies and small size is really important, you could also just write your script in a low level compiled language (C, Rust, Zig, …), link it statically (e.g. with musl) and execute that.
I use Lua for this sort of thing. Not my favorite language, but it works well for it. Easy to build for any system in the last 20-30 years, and probably the next 20 too. The executable is small so you can just redistribute it or stick it in version control.
If you can, just use Perl. Probably installed on your systems, even the ones without python.
So often the right answer, perl. It’s a shame that it’s so unfashionable these days.
Tried bash, Make, and awk/sed. All hit brick walls. Finally landed on pyinvoke. Two dependencies to install on any new machine. Never had problems. Also, easy to debug and modify as projects evolve.
I use just for that usecase - highly recommended
luajitis small, fast(well, it can jit), and has a small but complete standard library and can do FFI pretty easily, should be ideal for most homelab usecaseldd $(which luajit) linux-vdso.so.1 (0x00007ffee9dc7000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb4db618000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb4db613000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb4db5f3000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb4db3ca000) /lib64/ld-linux-x86-64.so.2 (0x00007fb4db799000)Realistically whatever problems you see in python will be there for any other language. Python is the most ubiquitously available thing after bash for a reason.
Also you mentioned provisioning scripts, is that Ansible? If so python is already there, if you mean really just bash scripts I can tell you that does not scale well. Also if you already have some scriptsz what language are they on? Why not write the function there?
Also you’re running syncthing on these machines, I don’t think python is larger than that (but I might be wrong).
Bro seriously just slap pyenv + pyenv-virtualenv on your systems and you’re good to go. They’re absolutely trivial to install. Iirc the latter is not a thing in windows, but if you’re stuck on windows for some reason and doing any serious scripting, you should be using WSL anyways.
English
Let alone the reliability, I somehow doubt my router would be able to run an LLM :)
Could use a hipster shell like fish, nushell or elvish. I know the latter two have the functional support you’re looking for.
