Skip to content
FraSharpPublic

About

A Unix shell for structured data, with native pipelines and POSIX script support

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

fshell (fsh)

a unix shell that passes structured data through pipes instead of text.

fshell keeps the syntax you already know (pipes, redirections, &&, globs, job control), but the values passed between commands have types: a number stays a number, a date stays a date, and a file listing is a list of records with named fields. you filter and sort by field name instead of splitting columns with awk, and @json, @csv and @table convert records to text and back.

existing bash and zsh scripts run through a bundled POSIX engine, so you don't need to rewrite them to switch.

status: work in progress. i use it every day, but bash and zsh have been tested for decades longer; the known gaps are listed in what's rough today.


examples

list the docs over 20KB, sorted by the biggest first:

$ ls docs | filter size > 20KB | sort size desc | map name size | @table
| name                 | size  |
|----------------------|-------|
| LANGUAGE.md          | 37898 |
| PERFORMANCE-AUDIT.md | 31129 |
| ARCHITECTURE.md      | 25986 |
| BUILTINS.md          | 21540 |

show the last three commits with their short hash, author and subject:

$ git log 3 | map short author subject | @table
| short   | author   | subject                                                 |
|---------|----------|---------------------------------------------------------|
| 40a3cf9 | FraSharp | docs: rewrite the readme around what fshell does today  |
| 05aa978 | FraSharp | tests: give the bare count test an empty standard input |
| a9ad276 | FraSharp | parser: report a missing closer at the end of input wh… |

a few more:

cat requests.ndjson | @json | filter status >= 500 | map path ms   # list the failed requests in a JSON log
git status | filter worktree == "modified" | map path                # list the files modified in the worktree
du -d 1 | sort size desc | limit 5                                   # show the five biggest directories here
df | filter use_percent > 90                                         # list the filesystems over 90% full
ps | filter cpu > 20.0 | sort cpu desc | @table                      # list the processes using more than 20% CPU

features

built-in commands that emit records

the common commands are built in. when their output goes into a pipeline they send records instead of formatted text; at the terminal they print the usual output.

command what it emits
ls name, type, size, last_modified, git_status, …
ps pid, ppid, user, cpu, rss, state, command, …
git status / log / branch / head changes, commits and branches, read from the repository without starting git
du / df disk usage per directory; size, free space and inodes per filesystem
files every file under a directory, like find
history every command you ran, with its exit code, duration and directory
doctor one record per check of your setup

any other git subcommand or flag (git commit, git push, git status --porcelain) runs the real git.

field completion

commands declare the fields they emit, so Tab completes field names: ls | filter <Tab> offers name, size, type and the rest, and git status | filter <Tab> offers the status fields.

the last result

at the prompt, _ holds the records the last command printed, so you can filter them again without running the command again:

du -s * | sort size desc
$_ | limit 5 | map path size

json and csv

@json and @csv parse text into records and write records back out as text. a JSON array, or a stream of one JSON object per line, becomes one record per item:

cat users.json | @json | filter active == true | map name email | @csv > active.csv
ps | filter cpu > 20.0 | @json        # one JSON object per line
echo $payload | json '.items[].id'    # select values with a jq-like path

@table, @bar, @yaml and @msgpack format records for output.

posix scripts

fsh --posix setup.sh                  # run a sh/bash script unchanged
sh {                                  # run POSIX code inside a .fsh script
    for f in *.log; do gzip "$f"; done
}
source --bash .venv/bin/activate      # source a bash script, e.g. a python virtualenv

set -e, set -u, trap, local, readonly and the other common builtins work in the POSIX engine. the compatibility notes list what still differs.

the .fsh language

anything you type at the prompt works in a .fsh script, and the other way round. the syntax is close to rust: types are inferred, and you can declare them where you want them checked.

fn deploy(service: String, port: Int) {
    echo "deploying {service} on port {port}"
}

let stage = "prod"

match $stage {
    "prod" => { deploy "api" 443 }
    _      => { deploy "api" 8080 }
}

errors are values you can catch and inspect, on exit runs a block when the script ends, and every repeats a block at an interval:

try {
    let port: Int = "8080"
} catch |err| {
    echo "bad config: {err.message}"
}

on exit { rm -f /tmp/deploy.lock }

every 5s { ps | sort cpu desc | limit 10 | @table }

interactive shell

  • completion menu: Tab opens a fuzzy, multi-column menu for files, commands, flags, variables, jobs and fields.
  • suggestions: grey text from your history and the filesystem while you type; Right accepts it.
  • history: Ctrl+R searches inline and Alt+H opens a full-screen explorer. history is also a command, so history --exit 1 --limit 10 lists your last ten failed commands as records.
  • alias expansion: an alias expands when you type the space after it, so you see the real command before running it. one Backspace turns it back into the alias.
  • git prompt: branch, changes and ahead/behind, read from the repository without running git status.
  • themes: 24-bit colour, several built-in palettes, and a full-screen editor for options, themes and aliases (config edit).
  • reload: reload --full restarts the shell binary and keeps your variables, functions, hooks, options and current directory.
  • configuration: all of the above is built into the binary, so there are no plugins to install. your own setup goes in ~/.config/fsh/init.fsh.

commands like rm -rf / ask for confirmation at the prompt and are refused in scripts.

the keybindings and widgets are documented in docs/WIDGETS.md.


goodies

did you mean

in the interactive shell, a mistyped command gets a suggestion you can run or edit straight away:

$ gti status
Command not found: gti. Did you mean 'git status'? Enter 'd' to run it, or 'e' to edit it.

errors you can read

errors point at the exact spot in your code and suggest a fix:

$ fsh build.fsh
FSH-PARSE-005

  × Expected expression, found ','
   ╭─[build.fsh:1:15]
 1 │ let x = [1, 2,, 3]
   ·               ┬
   ·               ╰── here
   ╰────
  help: Check the syntax around this position. Did you miss an operator or a value?

every error has a code: explain FSH-PARSE-005 describes it in more detail, and explain --list lists them all. fsh --error-format json prints errors as JSON, for editors and CI.

mark and grep

grep keeps only the lines that match. mark keeps every line and highlights the ones that match, so you can see a warning together with what happened before and after it:

$ cat build.log | mark warning
compiling fshell-core
compiling fshell-engine
> warning: unused variable `x`
compiling fshell-repl
finished in 41s

$ cat build.log | grep warning
warning: unused variable `x`

use grep to narrow the output down, and mark to read it in context.

bar charts

@bar draws a bar chart from any numeric field:

$ du -d 1 crates | filter path != "crates" | sort size desc | limit 4 | map path size | @bar
crates/fshell-repl      █████████████████████████████████████████████     1.5 MB
crates/fshell-builtins  ████████████████████████████████████     1.2 MB
crates/fshell-engine    ███████████████████████████████████     1.1 MB
crates/fshell-core      ████████████████████████████   942.1 KB

z and zi

z fshell jumps to the directory you visit most often whose path contains "fshell". zi opens a picker with all of them. both are built in, so you don't need zoxide.

history

every command is saved with its exit code, duration and directory, including the ones run by scripts and fsh -c. besides searching it, you can query it:

history --stats                       # totals, success rate, most used commands
history --cwd                         # commands run in this directory
history --exit 1 --limit 10           # the last ten failed commands

du

du reads sibling directories in parallel and counts hard-linked files only once. it reports sizes in 1K blocks on macOS too, the same as GNU du, and in a pipeline it emits one record per directory with its size, file count and directory count.


installation

requirements

  • Linux or macOS
  • Rust 1.99 or newer (rustup.rs)
  • a C compiler (on macOS, the Xcode command line tools)

build from source

git clone https://github.com/FraSharp/fshell
cd fshell
cargo install --path .                  # installs fsh into ~/.cargo/bin

there are no prebuilt binaries yet.

make it your login shell

command -v fsh                 # the full path to fsh
cat /etc/shells                # it has to be listed here; add it (as root) if it isn't
chsh -s "$(command -v fsh)"

log out and back in to try it. to go back, chsh -s /bin/zsh (or any shell you know is installed).


quick start

fsh                                       # interactive shell
fsh -c 'ps | filter cpu > 20.0 | @table'  # run one pipeline and exit
fsh deploy.fsh                            # run a .fsh script
fsh --posix setup.sh                      # run an existing sh/bash script

help lists every command and topic, and help filter explains one.


pipeline reference

these stages are part of the language and run inside the shell, without starting a process:

stage what it does example
filter <expr> keep items where the condition holds ls | filter size > 1MB
map <fields...> keep only these fields (a.b reaches inside), or compute new ones in parentheses ps | map pid (cpu / 100.0)
sort [field] [asc|desc] sort by a field ls | sort size desc
limit <n> keep the first n items ps | sort cpu desc | limit 5
count count the items ls | filter size == 0 | count
mark <pattern> highlight matching lines, keep the rest cat build.log | mark "WARN"
hash hash the whole stream, or each record cat archive.tar | hash

grep also works on records. size literals use SI and binary units: 10KB is 10,000 bytes, 10KiB is 10,240.

the full reference is in docs/PIPELINES.md.


documentation


what's rough today

fshell is a work in progress. these are the known gaps:

  • the POSIX engine isn't bash. most scripts run, but some corners still behave differently. for example, trap … ERR is accepted but its handler never runs.
  • json paths are a small part of jq. fields, indexes (negative ones count from the end) and [] work; filters, pipes and slices don't.

if one of these matters to you, open an issue. contributions are welcome.


contributing

see CONTRIBUTING.md for setup, tests, CI and pull requests.

cargo build --release
cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --check

license

GPLv3

About

A Unix shell for structured data, with native pipelines and POSIX script support

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages