A Forgejo-Runner with Podman on Alpine

2025-05-24T00:00:00+00:00

What this is about</h2>
This is a guide on how to install a Forgejo-runner on alpine Linux</a> using rootless Podman</a> running as its own user.

Don't let the length of this post scare you. most of it is explaining why this guide works which should make catching mistakes easier. </blockquote>

I'll assume you have root access to the machine to install dependencies, service scripts and for creating users. </blockquote>

The installation here will use the runit service supervisor, as it is available on every distribution and easy to learn. </blockquote>

This guide is structured, with a future you, who has multiple runners (i.e. for another codeforge, organisation, friends, …) in mind. You'll find hints for repeating this guide. </blockquote> </div>
We will:
Install the needed dependencies</a></li>
Add a user and configure subuid and subgid</a></li>
Set up Podman as a service</a></li>
Configure the runner</a></li>
Set up the runner as a service</a></li>
Test the runner</a></li>
Enable Podman in Podman (optional)</a></li> </ul>
Installing the Needed Dependencies</h2>
Community repository needed: Some of these packages come from the alpine community repository. Uncomment the relevant line in `/etc/apk/repositories</code> and run apk update</code> to enable it.`
apk add forgejo-runner podman slirp4netns runit sudo iptables</code></pre> If you want to use Podman inside your CI: Add the fuse</code> package if you want to enable Podman in Podman for your runner (i.e. for building containers). The slirp4netns</code> package is needed for networking with rootless Podman. sudo</code> is needed to switch users, we'll also use it as part of the runit services to make configuration easier. iptables</code> is needed by Podman for networking.
`Enabling Cgroups</h2> Cgroups are needed by container runtimes like Podman, to enable the on alpine:`
`# Start cgroups /etc/init.d/cgroups start # Make sure it gets started on every boot rc-update add cgroups # Should say that it is "started" /etc/init.d/cgroups status</code></pre> Note: Cgroups are a kernel feature, but alpine treats them as if they were a service to make them more convenient to manage.`
Starting runit</h3>
After installation we have to tell OpenRC to start runit and do that on every boot:
`# Start runit /etc/init.d/runitd start # Make sure it gets started on every boot rc-update add runitd # Should say that it is "started" /etc/init.d/runitd status</code></pre>`
Creating a User for the Runner</h2>
For the runner we'll create a regular user account without any special access. For this example I'm calling the user `runner-for-someone</code>, you can of course use your own name, make sure you always replace runner-for-someone</code> with that.`
adduser -D runner-for-someone</code></pre> The -D</code> option will disable password login for the just created user, there will be no need to log into it directly. Use a naming convention: Using the runner-for-…</code> naming scheme makes it easier to keep track of which user accounts are for runners and who the runners are for. It is not a requirement, but choosing a naming convention and sticking with it helps in the long run. Now the important part is to add subuids and subgids. Configure SubUIDs</h3> In the file /etc/subuid</code> add the following line to reserve 99999</code> user ids starting at user id 100000</code>. Important: If there are already UIDs reserved as subuids in that range, pick a multiple of 100000 where the next 99999 UIDs aren't reserved (i.e. 500000)). Why multiples of 100000? Reserving multiples of 100000 isn't a requirement, but it makes it very easy to keep track of reservations. runner-for-someone:100000:99999</code></pre>Configure SubGIDs</h3> And finally add the same line you added to /etc/subuid</code> to /etc/subgid</code> too. Is reserving the same range a requirement? Using the same range for SubUIDs and SubGIDs isn't required, but again, it makes it easier to keep track of allocations. Testing Podman</h2> sudo -iu runner-for-someone podman run --rm alpine echo "It works!"</code></pre> This should run the echo "It works!"</code> command inside a rootless container that runs under the runner-for-someone</code> user on the host. Note on the sudo invocation: The -i</code> flag means the command is ran as if it was for an interactive session, which sets up some extra environment variables that Podman needs. The -u runner-for-someone</code> part tells sudo to switch to the given user instead of root. If you get errors about insufficient SubUIDs and SubGIDs and have ran Podman before reserving the SubUIDs/SubGIDs the following command should make Podman aware of the change and fix it. sudo -iu runner-for-someone podman system migrate</code></pre> If you're here after creating a Podman service: Remember to restart Podman using sv restart /etc/service/runner-for-someone_podman</code> for the fix to take effect in the runner. Podman as a Service</h2> Now that we know, that Podman is working we can start it as a service using runit. Actually, to make your life easier should you want multiple runners (i.e. for yourself, an organisation, a second account, …), we'll create an easy to use template and then turn it into a service. One Podman service per user. I though Podman doesn't require a service? The forgejo-runner</code> wants to talk to Podman using a Unix socket, this is why wee need a Podman service. The Podman Service Template</h3> Note if you are creating your second runner: You only have to set up the templates once, if you already have set it up correctly you can skip to creating the service. Create the directory /etc/sv/generic-forgejo-runner-podman/</code> and the file /etc/sv/generic-forgejo-runner-podman/run</code> with the following content: #!/bin/sh # This parses the username from the directory name USERNAME="$( basename "$(pwd)" | cut -d "_" -f 1 )" # And this switches the user and runs the actual podman command exec sudo -iu "$USERNAME" \ sh -c 'exec podman system service --time=0 "unix://${HOME}/podman.sock"'</code></pre> Then run the following commands to make the script executable: chmod +x /etc/sv/generic-forgejo-runner-podman/run</code></pre>What the Service Template Does</h3> To explain what's going on (feel free to skip to the next headline): This template will be usable without any configuration, any service created from it should have a name that is the username it runs as, the first component followed by an _podman</code>. USERNAME="$( basename "$(pwd)" | cut -d '_' -f 1 )"</code></pre> This first line makes use of the fact that with runit we can rely on the working directory being the one that is named after out service. The basename "$(pwd)"</code> outputs the name of the directory we are in without the path and the cut -d '_' -f 1</code> cuts the name at the underscore (_</code>) and outputs what was before the underscore. exec sudo -iu "$USERNAME" \</code></pre> The sudo -iu "$USERNAME"</code> switches into the user account whose name ended up in the USERNAME</code> variable in the first line simulating an interactive login to get some extra environment variables that Podman expects to exist. The exec</code> at the start tells the shell to "replace" itself</a> with the command instead of starting it as a child process. This is the simplest way to ensure that stopping the service will actually stop it and not leave some detached processes running in the background. The backslash (\</code>) at the end indicates that the next line is a continuation of this one instead of a new command. It is just there for readability. sh -c 'exec podman system service --time=0 "unix://${HOME}/podman.sock"'</code></pre> This looks like a new command, but it is actually an argument to and ran by sudo in the context of the runner user. The sh -c</code> runs whatever comes next as if it was a little shell script, so the next argument is actually another command. exec podman system service --time=0 "unix://${HOME}/podman.sock"</code></pre> Again exec</code> at the start so that the shell replaces itself instead of getting in the way. And our actual podman system service</code> command that makes Podman reachable using a Unix socket in the home directory. The --time=0</code> tells Podman to not time out when there is nothing to do. The variable ${HOME}</code> gets evaluated now instead of in the previous steps because the whole command was wrapped in single quotes. This is important because only now $HOME</code> is set to the runner users home instead of /root</code>. Why do this detour instead of putting the socket inside /home/$USERNAME/podman.sock</code> even though that would have the same result? </blockquote> </div> Because /home/$USERNAME</code> is only correct in the usual case, if you want to put the runner homes somewhere else in the future this will save quite some time and headaches. </blockquote> Sometimes, being lazy requires a bit of effort. </blockquote> </div> And why $HOME/podman.sock</code> instead of the default path? </blockquote> </div> Because that puts it in plain sight, less "magic" to get confused by. Any other writable file path owned by the runner user would have worked too. </blockquote> </div> Creating the Actual Service</h3> Template done, now lets create a service: # Create the directory for the actual service. Note the USERNAME_podman pattern mkdir /etc/sv/runner-for-someone_podman # Take the run script from the template using a symbolic link ln -s /etc/sv/generic-forgejo-runner-podman/run /etc/sv/runner-for-someone_podman/ # And create a symbolic link in /etc/service to "enable" the service ln -s /etc/sv/runner-for-someone_podman /etc/service/ # The service should now be in an "up" state sv status /etc/service/runner-for-someone_podman</code></pre> You can control runit services using the sv</code> command and through a path to the directory that contains the run</code> script. Have a look at the sv</code> manual if you are not yet familiar with it </a> If you need it for debugging, you can always sv stop</code> your service and run the run</code> script like you would run any other shell script, make sure you are in the right directory so that you can run it as ./run</code> so that the username gets read correctly. Configuring the Forgejo Runner</h2> For this step witch into the runner user using the following command: sudo -iu runner-for-someone</code></pre>Getting a Registration Token</h3> Now is a good time to obtain a runner token … … for yourself: Log into the Forgejo instance using the website</li> Click on your avatar in the top right</li> Navigate to Settings</kbd></li> Actions</kbd> (in the Sidebar)</li> Runners</kbd></li> Create new runner</kbd> (Button in the top right)</li> </ul> </li> Copy the registration token</li> </ol> … for your organisation: Log into the Forgejo instance using the website</li> Navigate to the repository overview of your organisation</li> Click the Settings</kbd> link in the to right (Might be hidden inside a 3-dot menu)</li> Navigate to Actions</kbd> (in the Sidebar)</li> Runners</kbd></li> Create new runner</kbd> (Button in the top right)</li> </ul> </li> Copy the registration token</li> </ol> Leave that tab open, we will later use it to observe our runner. Registering the Runner</h3> Back in your shell run: forgejo-runner register</code></pre> It will interactively ask a few questions. Example output of a runner registration for the user slatian on codeberg.org.</figcaption> INFO Registering runner, arch=amd64, os=linux, version=6.3.1. WARN Runner in user-mode. INFO Enter the Forgejo instance URL (for example, https://next.forgejo.org/): https://codeberg.org/</kbd> INFO Enter the runner token: [… registration token redacted …]</kbd> INFO Enter the runner name (if set empty, use hostname: bad-wolf.slatecave.net): runner-for-slatian.bad-wolf.slatecave.net</kbd> INFO Enter the runner labels, leave blank to use the default labels (comma-separated, for example, ubuntu-20.04:docker://node:20-bookworm,ubuntu-18.04:docker://node:20-bookworm): [… this affects the `runs-on:` directive, leave it empty if you're just starting out …]</kbd> INFO Registering runner, name=codeberg-runner-for-slatian.bad-wolf.slatecave.net, instance=https://codeberg.org/, labels=[docker:docker://data.forgejo.org/oci/node:20-bullseye]. DEBU Successfully pinged the Forgejo instance server INFO Runner registered successfully. </samp> </figure> The resulting registration will be store in the file .runner</code>, you don't have to do anything with it, but you should know where your configuration is. If you refresh the browser window where you got the token from now, you should see a new runner that is marked a "Offline". (Leave the tab open, we'll need it again after creating the runner service) Generating the Runner Configuration File</h3> Speaking of configuration, your runner needs a configuration file. Generate one: forgejo-runner generate-config > ~/config.yml</code></pre> You don't have to change this configuration file either, but you should have a look at it anyway. The most interesting settings are: enable_ipv6</code> </dt> If you know you have working IPv6 set this to true</code>. </dd> force_pull</code> </dt> Set this to true</code> to make Podman check if a new version of an image is available every time it is needed. Podman is smart enough to not download an image again if it is already up to date. </dd> container.docker_host</code> </dt> Leave it set to -</code> or an empty string, the runner will then make use of the DOCKER_HOST</code> variable to find the socket, which we can set in the service file. </dd> privileged</code> </dt> Be careful with this one, if enabled container isolation becomes almost non-existent and everything that runs on you CI will have full access to your runner user. Leave it off. </dd> </dl> How to quickly find out if IPv6 is working? Test if ping v6.echoip.slatecave.net</code> or pinging any other IPv6 only host works. We are done with the runner user for now. exit</code></pre>The Runner Service</h2> Like with the Podman service this will be split into a template and an actual service. Again, resulting in one service per user. The Runner Service Template</h3> Create a directory at /etc/sv/generic-forgejo-runner-runner/</code>. Create a file /etc/sv/generic-forgejo-runner-runner/run</code> with the following content: #!/bin/sh USERNAME="$( basename "$(pwd)" | cut -d "_" -f 1 )" # Make sure the podman service is running sv start "../${USERNAME}_podman" # Start the forgejo runner exec sudo -iu "$USERNAME" \ sh -c 'DOCKER_HOST="unix://${HOME}/podman.sock" exec forgejo-runner daemon'</code></pre> Make sure the file is executable: chmod +x /etc/sv/generic-forgejo-runner-runner/run</code></pre>Explaining the Runner Service Template</h3> This works much like the template we used to start Podman, so I'll only go over the parts that are different. sv start "../${USERNAME}_podman"</code></pre> This will make sure the corresponding Podman service has started before starting the runner, even if it was stopped for some reason. sv start</code> will by default wait for up to 7 seconds which should be plenty of time for Podman to start. And inside the sh -c</code>: DOCKER_HOST="unix://${HOME}/podman.sock" exec forgejo-runner daemon</code></pre> This will set the DOCKER_HOST</code> environment variable and run the forgejo-runner</code> in its service mode. The exec</code> again are there to get the shells out of the way as soon as they've done their work. Not really a daemon: Historically the "daemon" mode made a program detach itself from the shells process tree and run in the background, which works great for unmanaged services, but actually gets in the way of supervisors like runit. The forgejo-runner daemon</code> command doesn't do that, contrary to what the name might suggest. Creating the Actual Runner Service</h3> Almost the same commands as with Podman to turn the template into a running service: # Create the directory for the actual service. Note the USERNAME_runner pattern mkdir /etc/sv/runner-for-someone_runner # Take the run script from the template using a symbolic link ln -s /etc/sv/generic-forgejo-runner-runner/run /etc/sv/runner-for-someone_runner/ # And create a symbolic link in /etc/service to "enable" the service ln -s /etc/sv/runner-for-someone_runner /etc/service/ # The service should now be in an "up" state sv status /etc/service/runner-for-someone_runner</code></pre>Testing the Runner</h2> Refresh the tab with the runner overview, the status should have changed from "Offline" to "Idle". Now create a new Forgejo repository, make sure the actions tab is enabled, if not click on Settings</kbd> in the upper right, navigate to Units</kbd>, </kbd>Overview</kbd>, enable the checkbox labelled Actions</kbd> and click Save</kbd>. Create a file in .forgejo/workflows/test.yaml</code> in the repository: --- on: push: jobs: test: runs-on: docker container: image: alpine steps: - run: echo "Hello World!"</code></pre> Commit and push it. In the "Actions" tab there should now be an action that is either waiting or already running. Don't worry if it takes up to a minute for the runner to pick up the job. After a few seconds everything should go green with a "Hello World!" in the logs. If it doesn't there is usually a hint hidden in the logs as to what went wrong. If it fails with "failed to create container": This one is misleading, try to sudo -iu runner-for-someone podman pull alpine</code>, it'll output a more helpful error message. (Also have a look at testing Podman</a> again.) 🎉 Congratulations, you now have a working runner. </blockquote> You should keep the test repository, it'll stay useful. </blockquote> </div> If you now want to try out some premade using</code> actions remove the image: alpine</code> line. Using alpine Linux as a base image is great if you know exactly what you need and want an efficient CI. But most premade actions unfortunately rely on the Ubuntu based nodejs image, which is the reason for that one being the default. Enabling Podman in Podman</h2> To enable Podman in Podman functionality without having to fall back on dropping all isolation the containers need access to fuse</code> (install and modprobe</code></a> it if you haven't done already) and some additional capabilities</a> to be able to create a container environment inside the container. What is Fuse? fuse</code> is a kernel subsystem</a> that allows mounting filesystems without any special privileges in a safe way. Podman uses this to assemble the filesystem visible from inside the container. The Forgejo runner doesn't directly support this way of running containers yet, but since we have a user just for the runner configuring Podman directly works too: # Install fuse apk add fuse # Make it useable immeadeately modprobe fuse # Add fuse to the list of services to be started on boot rc-update add fuse # switch to the runner user sudo -iu runner-for-someone # create the directory for the configuraion file mkdir -p ~/.config/containers/</code></pre> Then create the file ~/.config/containers/containers.conf</code>: [containers] devices = ["/dev/fuse"] default_capabilities = [ "SYS_ADMIN", "MKNOD", {append=true} ]</code></pre> And to test if that configuration file worked: podman run --rm quay.io/podman/stable podman run alpine echo Hello Podman</code></pre> If it didn't work you'd either get a message that mentions the absence of fuse or "potentially insufficient UIDs or GIDs" if the SYS_ADMIN</code> capability is missing. To also enable this feature in the runner you have to restart the Podman service: # Back to root exit # Restart the podman serivce sv restart /etc/service/runner-for-someone_podman</code></pre> If you want to know what exactly why those are necessary I highly reading the ultimate "Podman inside a container" guide by Dan Walsh and Urvashi Mohnani on the Redhat blog: How to use Podman inside of a container </a> Maintenance</h2> It is a good idea to get rid of old images from time to time, you can use the following commands to manage Podman images for your runner: sudo -iu runner-for-someone # list images podman images # remove outdated or no longer needed images podman image rm <image-id-goes-here></code></pre> Another thing to check for are stopped containers that never got cleaned up, this may happen because of sudden shutdowns or by forgetting the --rm</code> on containers during testing. sudo -iu runner-for-someone # list running AND stopped containers podman ps --all # remove old containers podman rm <container-id-goes-here></code></pre>Other Resources</h2> Forgejo Actions user guide</a></li> Forgejo Actions administrator guide</a></li> Forgejo Runner installation guide</a></li> Using Forgejo Actions (Self-hosted) by Codeberg</a></li> </ul> The Beginning …</h2> Now you have your own runner (hopefully 😅), it's up to you what you want to with it. </blockquote> I highly encourage you to keep the test repository around, try some unconventional things, break it, fix it, DIY some actions</a> instead of importing and learn what makes the actions run. </blockquote> Another thing to try is building your own images. </blockquote> Whatever you do, have fun and make the world a nicer place to be! </blockquote> </div> License Note</h2> This article is licensed under CreativeCommmons BY-SA 4.0 International</a> (this is different from the usual CC BY-NC-SA here). If possible please link back to this article.

xdg-settings: Setting a default browser isn't that simple

2025-01-08T00:00:00+00:00

What is xdg-settings?</h2>
`xdg-settings</code> is part of the` `xdg-utils</a> and according to its manpage</a> it can query, check for and set different properties of an XDG-desktop.`
`Running xdg-settings --list</code> reveals that it only ever got implementations for setting the default web browser and for setting URL-scheme handlers (which is the more generic case of setting a default web browser).`
The reason I started looking into this is because someone opened an issue</a> about xdg-open</code> ignoring the BROWSER</code> variable, which turned out to be xdg-settings</code> giving out misinformation about the order of variables. </blockquote> This is good by the way! if you notice something broken with a tool open an issue, even if you are wrong about the part that is broken. Someone will help figuring out the actual issue, especially in systems with multiple components where a weirdness in one place can be the symptom of an error somewhere else. </blockquote> </div> I'll take a look at the current git HEAD with the commit hash 0f6385262417f1c0c4d13bc05d95c32578272b64</code>, though xdg-settings</code> hasn't been touched for years, so this also applies to the 1.2.1</code> version you probably still have. (except for Lxqt, that one is new) xdg-settings makes use of some common functionality I already described when taking apart xdg-open</a>. Overview over xdg-settings</h2> Inside xdg-settings, it is split up into multiple sections. That is helper functions for browser setting, MIME related functionality, desktop specific functionality. Desktops that have their own sections are: KDE</li> Deepin</li> GNOME (Gnome 2 that is)</li> GNOME 3.x</li> Lxqt</li> Xfce</li> generic</li> </ul> While browsing the sections one will also notice common function prefixes: get_browser_</code></li> check_browser_</code></li> set_browser_</code></li> get_url_scheme_handler_</code></li> check_url_scheme_handler_</code></li> set_url_scheme_handler_</code></li> </ul> There is also a big dispatch function (dispatch_specific</code>) that calls the appropriate function based on the given handler and the requested command. The handler is derived based on the detected desktop environment (see the xdg-open article on desktops to learn how that works</a>) in a switch statement right at the end of the file. The desktops are translated as follows: Desktop</th> Handler</th></tr></thead> kde</td> kde</td></tr> deepin</td> deepin</td></tr> gnome</td> gnome</td></tr> gnome3</td> gnome3</td></tr> cinnamon</td> gnome3</td></tr> lxde</td> lxde</td></tr> mate</td> mate</td></tr> lxqt</td> lxqt</td></tr> xfce</td> xfce</td></tr> generic</td> generic</td></tr> enlightenment</td> generic</td></tr> </tbody></table> wsl</code>, cygwin</code>, darwin</code> and flatpak</code> error with "unknown desktop environment". Listing the supported settings</h2> Listing the supported settings is implemented in an interesting way. The dispatch function contains multiple switch-case statements for the supported parameters, one of each line is annotated with comment, #PROP: {description goes here}</code>. The list function greps the file for line containing that comment and then uses some regex magic to format that into a human readable list. The helper functions</h2> get_browser_mime <mimetype></code> </dt> A wrapper around xdg-mime query default $1</code> that defaults to text/html</code>. </dd> fix_local_desktop_file <desktop-file-name> <mimetype></code> </dt> If a desktop file exists locally for the user make sure it has a given mime-type in its list of supported mime types. </dd> set_browser_mime <desktop-file-name> <mimetype></code> </dt> This attempts to undo any desktop changes using fix_local_desktop_file</code> and then uses xdg-mime default</code> to set the default browser, checks if the change happened and sets it back if the change didn't have the desired effect. </dd> </dl> Generic Browser configuration</h2> The generic browser configuration makes use of the already mentioned helper functions. Getting the default Browser</h3> This is implemented in get_browser_generic</code>. If the BROWSER</code> environment variable is set, the script tries resolving the command to a desktop file, and if successful outputs that as the default browser. If not the script continues. If the mimetype x-scheme-handler/http</code> resolves to a desktop file that is returned as the default browser. If not the script continues. Bug Note: The order of first checking the BROWSER</code> variable and then checking the mimetype is reversed from how xdg-open</code> reads them. It also puts a less granular configuration mechanism over a more granular one. I consider this a bug. If the binary x-www-browser</code> resolves to a desktop file that is returned. Otherwise the script exits wit exit code 4</code> (Action failed) Checking the default Browser</h3> Checking the default Browser is distinct from getting and comparing the default browser in that instead of checking the first setting that gives a sensible value it checks all relevant settings and only returns true if all of those settings apply. This is implemented in check_browser_generic</code>. There are three functions that have to resolve to the given desktop file before the check function says yes</code>, that is the default browser: The browser getting function get_browser_generic</code></li> Resolving the handler for the text/html</code> mimetype</li> Resolving the scheme handler for https</code></li> </ul> Setting the default Browser</h3> This is implemented in set_browser_generic</code>. Bug Note: This function errors if the BROWSER</code> environment variable is set, this is related to xdg-settings</code> considering BROWSER</code> to be higher priority than the more granular x-scheme-handler/</code> mimetypes. After testing for the BROWSER</code> environment variable the set function tests if the desktop file resolves to a binary, this makes sure that the requested file exists and has a non-empty Exec</code> key which is the minimum needed to actually open URLs. It the n uses the set_browser_mime</code> to set the following mimetypes to be handled by the requested desktop file: text/html</code></li> x-scheme-handler/http</code></li> x-scheme-handler/https</code></li> x-scheme-handler/about</code></li> x-scheme-handler/unknown</code></li> </ul> Generic URL scheme configuration</h2> Generic URL scheme configuration is mostly wrapping around the already described helpers with some special treatment for the http</code> and https</code> schemes when the BROWSER</code> environment variables are set. Getting the scheme handler</h3> This is implemented in get_url_scheme_handler_generic</code>. Passes the call through to the get_browser_mime</code> helper with the mimetype x-scheme-handler/{scheme}</code>. Except when BROWSER</code> is set and the scheme is either http</code> or https</code>, then it calls get_browser_generic</code>. Checking the scheme handler</h3> This is implemented in check_url_scheme_handler_generic</code>. Tests if the desktop file resolves to a binary and if so it tests if it is returned by get_url_scheme_handler_generic</code>. Setting the URLs scheme handler</h3> This is implemented in set_url_scheme_handler_generic</code>. Bug Note: This function again has special handling for BROWSER</code> being set the scheme being http</code> or https</code> that it refuses to work with an error message that it can't change the BROWSER</code> variable. If the desktop file resolves to a command it uses set_browser_mime</code> with x-scheme-handler/{scheme}</code> as the mimetype. To be continued …</h2> This post won't cover the special cases for each desktop. I'm taking a slow start to 2025 and will describe those in a followup post. Reading through xdg-settings</code> really shows that it hasn't been maintained for a while (which isn't surprising given that the xdg-utils were effectively abandoned for multiple years). This is a call to action and for help: See if your favourite desktop is implemented correctly in xdg-settings</code> and if not open an issue</a> or merge request</a>. I'll help you if you want</a>. Thanks for reading, I hope you had an incident free calendar rollover! Why not use && as a shortcut in shell scripting 2024-10-07T00:00:00+00:00 What am I Talking About</h2> If you've been shell scripting for some time you have most probably seen the following construct: [ -n "$something" ] && foo</code></pre> This runs the foo</code> command, but only if $something</code> isn't empty, a very useful and common thing to check for in some scripts. So far no problem. Another useful thing in scripts is set -e</code> which aborts the whole script if some "line" (as in collection of commands) returns with a non-zero exit code. It makes sure that a shell script exits instead of causing havoc. The problem now arises when one combines the two. Now if $something</code> isn't empty everything works fines as before, but if it is empty, the test returns an error code, which gets treated as a signal to stop the script. Not good. How to Fix it the Verbose Way</h2> One place a command is allowed to fail is inside the condition of an if</code> statement, pretty obvious when one says it out loud, because that is literally the job of the if</code>, redirect the flow of execution based on whether a command was successful or not. So the above can be rewritten to: set -e # … if [ -n "$something" ] ; then foo fi</code></pre> Which works as intended again, but is kind of bulky. The Working Shortcut</h2> A way of using a shortcut with set -e</code> is rephrasing the problem so that the test returning false gets caught. For this to happen one has to invert the condition and catch when it fails, luckily most commands used in such shortcut condition usually also support testing for the opposite. Manual for test</code> and [</code> </a> So that the initial construct can be rewritten as: [ -z "$something" ] || foo</code></pre> The -z</code> tests is a string is empty, so the exact opposite of -n</code>, if the test for emptiness fails the command gets run. An easier way to read this is: Either $something</code> is empty (happy) or we run foo</code> (also happy) </blockquote> That's it!</h2> This of course depends on foo</code> succeeding, which I assumed the whole time for readability. If your shellscript also makes that assumption it is usually a very good idea to use set -e</code>. Also this doesn't affect what you use inside the condition of an if</code> statement, there &&</code> is completely fine. Thanks for reading! I know it has been a while. </blockquote> If you want to learn more, see my other shell articles</a>. </blockquote> </div> Portable Shell-Scripting</a></li> Shell Null Termination / Separation</a></li> </ul> Fixing PWM flicker on the Arduino UNO R3 2024-04-08T00:00:00+00:00 What happened?</h2> Since I'm trying to tell a story here, let's start at the beginning. I have an Alarm clock</a> that reliably wakes me up every workday, featuring a piezo beeper and an RGB backlit LCD connected to an ATmega328P Microcontroller that cosplays as an Arduino UNO R3. The tone()</code> function</a> messes up the PWM on the Arduino Pins 3 and 11. (They are using the same timer that also generates the beeper frequency.) No problem I though, I'll hook up my RGB backlight up to Pins 5, 9 and 10 … Note: My previous setup had one LED on pin 3 instead of pin 5 because I wasn't aware of the tone()</code> functions side effects when building the hardware. The Problem</h2> The problem was now that the display seemed to be flickering, turning up one colour at a time to the maximum doesn't result in flicker. My first suspicion was a software bug … Nope, cant be — setting the LEDs just once a second also makes it flicker. Also the flicker only occurs when the LED on pin 5 was involved, mixing the LEDs on pins 9 and 10 worked fine. So looking up the Pinout</a> which I was reminded that there are three timers inside the ATmega328P, each capable of driving two PWM channels. Pins 9 and 10 are on timer 1, while pin 5 is on timer 0 … suspicious. The obvious difference was that one uses an 8-bit counter and the other 16 bit one, so maybe some thing wrong with the 16-bit timer? (Nope.) At this point I found an answer on the Electronics stack exchange</a> pointing out that what I already knew as timer 0 runs at a different PWM frequency. Second Theory: They are not on the same clock and I see some fancy interference pattern … So will the flickering when I reprogram timer 0 to the same setting as the others? </blockquote> Some digging in the Arduino code I found where it configures the timers</a> with a helpful comment: on the ATmega168, timer 0 is also used for fast hardware PWM (using phase-correct PWM would mean that timer 0 overflowed half as often resulting in different millis() behaviour on the ATmega8 and ATmega168) </blockquote> and a bit further down the code: timers 1 and 2 are used for phase-correct hardware PWM this is better for motors as it ensures an even waveform </blockquote> The ATmega168 is the predecessor and quite similar to the ATmega328P so I guessed that it still applies. With timer 0 being responsible for the correct behaviour of millis()</code> reconfiguring it isn't a path I want to go down. But can I switch timer 1 to that non-phase correct mode that timer 0 is using? If you want to follow the code: The cbi()</code> and sbi()</code> functions clear and set bits in values, in this case the processors registers. All the cryptic symbols of upper case characters and numbers are the names of registers and bits and explained in the Datasheet. Consulting the ATmega328P Datasheet (PDF)</a> to find out what all the options mean: Both timers are on the System clock divided by 64</li> timer 1 is in Phase correct mode (already knew that)</li> timer 1 is using only 8 of its 16 bits</li> timer 0 is in a mode called fast PWM, which just means really simple and barebones PWM</li> The Fast-PWM effectively doubles the PWM frequency</li> </ul> The Solution</h2> So the only difference is one Timer is in Fast PWM and the other is in Phase Correct PWM. If you skipped here: The ATmega328P in the UNO R3 has three timers, PWM pins 5 and 6 are hooked up to timer 0 which is in a different mode than the other timers for the millis()</code> function to work. So lets try the pretty obvious and switch timer 1 to Fast PWM, which is just one bit according to the Datasheet. So I add the following code to my setup()</code> function: // Set timer/counter 1 from 8-bit phase correct to 8-bit fast bitWrite(TCCR1B, WGM12, 1);</code></pre> And … the flickering is gone. Problem solved. If you are now interested in more details on how PWM works and what those timers actually do: ATmega328P: PWM and Timer/Counters </a> Why it flickers?</h2> Two PWM signals one running at double the frequency of the other shouldn't noticeably interfere when the slower one is running at 490Hz. The problem is that they aren't really double with how the two involved modes work. The Fast PWM mode simply counts up and wraps from 255 to 0, making one iteration 256 cycles long. The Phase Correct PWM mode counts up until 255 and then starts counting down again, but to arrive at 0 it takes 510 cycles. An off by two error resulting in not quite double the frequency. The problem can be illustrated with a 2 bit counter. The second time the Fast mode finishes a full iteration the Phase Correct Mode is already two cycle into the third.</figcaption> Fast</th> 0</td> 1</td> 2</td> 3</td> 0</td> 1</td> 2</td> 3</td> 0</td></tr> Phase Correct</th> 0</td> 1</td> 2</td> 3</td> 2</td> 1</td> 0</td> 1</td> 2</td></tr> </table> (Count the steps between the numbers, those are the cycles, not the numbers themselves)</figcaption> </figure> So the actual frequencies are: Mode</th> Frequency</th> Calculation</th></tr></thead> Fast</td> 976.6Hz</td> 16Mhz / 64 / 256</code></td></tr> Phase correct</td> 490.2Hz</td> 16MHz / 64 / 510</code></td></tr> </tbody></table> If you were to plot those two frequencies overlapping each other you will notice that the interference matches a 3.8Hz wave, which should roughly match the low frequency flicker you were observing. </picture> </a> Group based Permissions using polkit 2024-01-09T00:00:00+00:00 What and why?</h2> I'm the kind of creature that really likes NetworkManager and avoids systemd when possible. The problem arises that, when one isn't using (e)logind things like being able to configure networks don't work anymore. Reason being that polkit is configured to only allow these actions for logged in sessions, but nothing told it that a session is active. Luckily those rules are not set in stone and one can reconfigure polkit however they like. Note: NetworkManager only supports polkit for configuring who can do what. This is okay because it is supposed to manage networks, not permissions. For my purposes I want to be able to send basic commands to NetworkManager because I'm in the wheel</code> (aka. sudo</code>) group, independent of me being logged in. Update on 2024-04-18: Previously I forgot to include the org.freedesktop.NetworkManager.settings.modify.system</code> action which prevented connecting to new wireless networks. Understanding polkit</h2> This was the first time I messed with polkit which is very well documented in the polkit(8) manpage</a>. Polkit is a service with its only task being to answer one question Is subject allowed to do action? </blockquote> The subject is roughly equivalent to a user. The action is a more or less rough description of what the subject wants to do identified by an id. Actions are described by XML-Files in /usr/share/polkit-1/actions/</code>. These contain localised names, icon-names and some sane default permissions for each action. In addition to actions and the default permissions, polkit supports rules written in JavaScript. These live in files in /etc/polkit-1/rules.d/</code>. Note: There is also /usr/share/polkit-1/rules.d/</code> but that one is intended for packages. As an admin /etc</code> is the preferred place. NetworkManager rules</h2> Before writing the rule I need the action ids that are used to configure NetworkManager. These can be found by searching the /usr/share/polkit-1/actions/org.freedesktop.NetworkManager.policy</code> file for all rules that work without any extra authentication given an active session. The results are: org.freedesktop.NetworkManager.enable-disable-network</code></li> org.freedesktop.NetworkManager.enable-disable-wifi</code></li> org.freedesktop.NetworkManager.enable-disable-wimax</code></li> org.freedesktop.NetworkManager.enable-disable-wwan</code></li> org.freedesktop.NetworkManager.network-control</code></li> org.freedesktop.NetworkManager.settings.modify.own</code></li> org.freedesktop.NetworkManager.settings.modify.system</code></li> org.freedesktop.NetworkManager.wifi.scan</code></li> org.freedesktop.NetworkManager.wifi.share.open</code></li> org.freedesktop.NetworkManager.wifi.share.protected</code></li> </ul> Note on inactive sessions: Even if an action is allowed for inactive sessions it isn't allowed when that session state can't be determined which is the case when (e)logind is missing. Polkit debug mode</h2> To save you some frustration with debugging: Polkit has a debug mode which is probably turned off when launched through your service manager of choice using the --no-debug</code> option. To get the debug output. Stop the polkit</code> service</li> Run /usr/lib/polkit-1/polkitd</code> as root in a terminal</li> Start the service again when done with debugging</li> </ol> Note: Polkit will drop privileges automatically after setting itself up. Adding a rule</h2> According to the manual I should be able to just place a .rules</code> file in /etc/polkit/rules.d/</code> and polkit will pick it up automatically. If you have polkit in debug mode you'll even get feedback whether your rules are valid JavaScript or not. For the NetworkManager scenario my script adds a rule function that first checks if the action id is in an array of allowed actions, then checks if the subject is in the wheel</code> group and if so returns that the action is allowed. If the rule doesn't return a result polkit will fall back to other rules and eventually the defaults. Translated to JavaScript the result is the following. polkit.addRule(function(action, subject) { var allowed_actions = [ "org.freedesktop.NetworkManager.enable-disable-network", "org.freedesktop.NetworkManager.enable-disable-wifi", "org.freedesktop.NetworkManager.enable-disable-wimax", "org.freedesktop.NetworkManager.enable-disable-wwan", "org.freedesktop.NetworkManager.network-control", "org.freedesktop.NetworkManager.settings.modify.own", "org.freedesktop.NetworkManager.settings.modify.system", "org.freedesktop.NetworkManager.wifi.scan", "org.freedesktop.NetworkManager.wifi.share.open", "org.freedesktop.NetworkManager.wifi.share.protected" ] if (allowed_actions.indexOf(action.id) >= 0) { if (subject.isInGroup("wheel")) { return polkit.Result.YES; } } })</code></pre>Other Things polkit Rules could do</h2> By testing for an action id prefix and (non-)membership of a group you could also deny access to a service by returning polkit.Result.NO</code>. Polkit can also call external commands and make a decision based on success or failure of a command. The manual has very good examples. </a> Bye</h2> I hope that was a useful bit of information about a thing that mostly stays out of the way and comes with an extra bit of suprise when it doesn't work. Pretty Atom-Feed Previews with XSLT 2023-12-22T00:00:00+00:00 Atom and XSLT what?</h2> I'll assume you already know what Atom-Feeds</a> and XML are and that you have some HTML skills. As an Atom-Feed is simply an XML-File on the web-server it makes sense linking to it</a>, unfortunately the Atom-XML in the Browser isn't pretty to look at. This is where XSLT (Extensible Stylesheet Language Transformation)</a> steps in. XSLT is a standardised way to turn an XML document into another and, as it turns out it is built into every major Web-Browser</a>, meaning it works perfectly well with my static site setup. This way one can deliver a standards compliant Atom-Feed that gets rendered into a preview for humans when viewed with a Browser. Awesome, lets do that! Note: If you've ever seen the message This XML file does not appear to have any style information associated with it. The document tree is shown below.</q> the Browser means XSLT, not CSS. Building an Atom-Feed with a Stylesheet</h2> To get the Browser to load a stylesheet one has to add tell it where it is by adding a line like following:  <?xml-stylesheet type="text/xsl" href="/assets/site_slatecave/atom_to_html.xslt"?> </code></pre> For the slatecave I've added the line to my atom feed template</a> and put a very minimalistic stylesheet into the static files. Note: I didn't have this template before this project so I customised Zola's builtin template</a> a little. Note: If you want to use XSLT 2.0 functions (mainly replace()</code> and tokenize()</code>) you have to set version</code> to 2.0</code> on the xsl:stylesheet</code> tag. A very minimal XSLT-Sheet that results in a hello-world message.</figcaption> <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="html" encoding="utf-8" indent="yes" /> <xsl:template match="/"> <html lang="en"> <body> <p>Hello from the XSLT-Sheet!</p> </body> </html> </xsl:template> </xsl:stylesheet></code></pre> </figure> Getting Content from the Atom-Feed</h2> To get content from the original XML one can use an <xsl:value-of select="/xpath/goes/here" /></code> tag. This tag replaces itself with the content of the first tag returned by the XPath in the select</code> attribute. What is XPath? XPath is a way to select tags from an XML Document, it works a bit like a Filepath and a bit like a CSS Selector. See my XPath Cheatsheet. </a> To spare you the headache: Don't forget to declare and use the atom</code> namespace on the XSLT-Sheet. The new opening tag of the XSLT-Sheet.</figcaption> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:atom="http://www.w3.org/2005/Atom"></code></pre> </figure> Nodes in the Atom feed can then be accessed like the following: <xsl:value-of select="/atom:feed/atom:title" /></code></pre> With that out of the way you follow the the w3schools XSLT Tutorial</a> to create such a sheet yourself. The slatecave.net Atom to HTML XSLT-Sheet </a> Content Security Policy</h2> After Uploading I noticed that it isn't working. A quick look at the debugging tools reveals that an XSLT sheet is treated like a script which was disabled, because the slatecave doesn't use any JavaScript. Setting script-src 'self';</code> allows loading and executing the XSLT-Sheet resulting in some pretty Atom-Feeds</a>. That's it!</h2> I hope that information was useful or at least interesting to you! This is more a braindump style of post, let me know how that went. xdg-mime: Mapping Files to Applications taken apart 2023-10-03T00:00:00+00:00 last time we took apart xdg-open</code>, the freedesktop file-opener and found out that xdg-mime</code> provides the mapping from file to application. So let's take it apart and see what's inside … What can xdg-mime do?</h2> xdg-mime</code> is a shell-script that helps with mapping files to the applications that open them. It implements querying a files mime-type, setting and getting the default application for a given mime-type as well as installing and uninstalling mime-type descriptions. The code is developed in gitlab.freedesktop.org/xdg/xdg-utils</a>. You can use xdg-mime --help</code> or man xdg-mime</code> yourself to get more details. xdg-mime query filetype <file></code> </dt> This sub-command prints the detected mime-type of a given file. (i.e. text/plain</code>) </dd> xdg-mime query default <mimetype></code> </dt> This prints out the name of the desktop file that is configured as the default opener for the given mime-type. (i.e. slatectx-app-kakoune.desktop</code>) </dd> xdg-mime default <applications> <mimetype> ...</code> </dt> Attempts to set the default opener for the one or more given mime-types to the given application which is identified by the name of its desktop file. </dd> xdg-mime install mimetypes-file <file></code> </dt> Installs a mime-type description file that follows the freedesktop mime associations specification</a> where it can be used to identify files. Can be put in user</code> or system</code> mode using the --mode</code> flag. </dd> xdg-mime uninstall mimetypes-file <file></code> </dt> As it says on the tin: It uninstalls a previously installed mime-type file. </dd> </dl> Note: This article ended up being quite long, since it should still be understandable when read in a non-linear manner feel free to skip sections that don't seem interesting. What is in the Script?</h2> Like the other xdg-utils xdg-mime</code> - in addition to a lot of custom logic - imports functionality from the xdg-utils-common.in</code>. Noteworthy imports from the common file are: a way to detect which desktop one is currently using</li> mapping the desktop file to the binary it calls and vice versa</li> helpers for command-line and UI</li> </ul> Unique to xdg-mime</code> is: a callback to update desktop specific mime database caches.</li> querying and setting of default applications from gnome, KDE and generic desktops</li> a function for getting desktop files matching a given mime-type.</li> functionality for figuring out an application when no explicit default has been set</li> functionality for opening the default application for KDE, gnome and generic desktops</li> command line parsing and validating</li> XML parsing in awk … okay</li> lots of parsing and updating files with awk</li> </ul> So this script seems to mostly do the work of parsing and writing the mime to application mapping database, which is to be expected, at first glance mime-type detection seems to be delegated to another command and is in here to provide a stable interface that applications can rely on while xdg-mime</code> does the figuring out how to derive the mime-type. xdg-mime</code> integrates with the KDE and Gnome mime databases, because those have been there before freedesktop.org</a> and the xdg-utils were a thing. More recently Lxqt made their own tool called qtxdg-mat</code></a> that has been integrated into xdg-utils</a>. Actions and CLI Plumbing</h3> Since calling xdg-mime</code> is based on sub-commands, some with their own custom options, this results the parser being a lot of nested case</code> statements and loops for parsing the extra options and arguments. To keep the logic from becoming an indentation mess xdg-mime</code> has a concept of actions. That is, that the command gets translated into an action-name that is written to the action</code> variable. Which is then reused for deciding what to do. Potential values for the action</code> variable are: info</code> </dt> for the filename to mime-type query. with the xdg-mime query filetype</code> sub-command. </dd> makedefault</code> </dt> for setting default mime-handlers with the xdg-mime default</code> sub-command. </dd> defapp</code> </dt> for querying the default application with the xdg-mime query default</code> sub-command. </dd> install</code> </dt> for installing mime descriptions with the xdg-mime install</code> sub-command. </dd> uninstall</code> </dt> for uninstalling mime descriptions with the xdg-mime uninstall</code> sub-command. </dd> </dl> Getting the MimeType of a File</h2> The info</code> action triggers a desktop detection and then decides which mime-type detection method to use. On success the output must be the mime-type of the file followed by a newline without any extra information. If the desktop is KDE and /usr/bin/file</code> is not an executable use info_kde</code>.</li> If the desktop belongs to the gnome gio family (Gnome,Cinnamon,Lxde,Mate,Xfce) use info_gnome</code></li> In case of Lxqt use info_lxqt</code> (new in the beta 1.2.0 release)</li> Otherwise use info_generic</code></li> </ul> Lxqt MimeType Query</h3> For Lxqt qtxdg-mat mimetype <file></code> is used to query the mime information of a given file. KDE MimeType Query</h3> The info_kde</code> checks the KDE_SESSION_VERSION</code> environment variable. For KDE 4 and 6 it uses the kmimetypefinder</code> command, for KDE 5 kmimetypefinder5</code> and for older versions it gets the mime-type from kfile</code>. Extra information is cut away using POSIX tools. Gnome MimeType Query</h3> The info_gnome</code> uses a mechanism similar to file opening where it uses a fallback cascade of multiple methods for detecting the mime type, using the first one that is available. gio info <file></code></li> gvfs-info <file></code></li> gnomevfs-info --slow-mime <file></code></li> </ol> Like with the KDE method any extra information is discarded using standard POSIX tools. Generic MimeType query</h3> The info_generic</code> also uses a little fallback cascade. mimetype --brief --dereference <file></code></li> /usr/bin/file --brief --dereference --mime-type <file></code></li> </ol> For both commands cutting away extra information is not needed. Note on /usr/bin/file</code>: It seems strange that /usr/bin/file</code> is used here instead of just calling the file</code> command. After finding no information on it and asking the original developers (this quirk has been around since 2006</a>) it turns out that nobody knows why the absolute path was used here, it would most probably not make a difference to change it back. How xdg-mime figures out the Default Application</h2> This functionality hides behind the defapp</code> action and can be triggered using the xdg-mime query default <mimetype></code> command. As a reminder this returns the name of the desktop-file describing the application configured as the default opener or an application that seems the best fit according to some simple heuristics in case no default is was explicitly set. This time there are only three implementations, a generic one, a new one for Lxqt and one for KDE below version 6. Note: The application returned by xdg-mime</code> is not necessarily the one that will open when calling xdg-open</code> on the same file</a>, mostly because xdg-open</code> delegates to openers which can have custom logic. KDE Default Application Querying</h3> KDE has a subsystem called Trader</a> which handles delegation of actions between various KDE components (services). To use it xdg-mime</code> invokes ktraderclient</code>, ktraderclient5</code> or ktradertest</code> (for older than KDE 4) as $KTRADER</code>. Asking it for a matching Application</code> service for the given mime-type, extracting the result from the returned text. If no ktrader command appropriate to the current KDE version can be found it falls back to the generic implementation. Lxqt Default Application Querying</h3> Again qtxdg-mat</code> is used: qtxdg-mat defapp <file></code></pre>Generic Default Application Querying</h3> For where there is no desktop specific mechanism, xdg-mime</code> has a generic, specification compliant 🥳 way to get the default application using the defapp_generic</code> function. Desktop Prefixes</h4> Before getting into querying default applications, first imagine someone switches between multiple desktops, maybe mate</code> and kde</code>, while you ave some applications you want to be the default everywhere (i.e. you preferred web-browser) other preferences might depend on the desktop you are currently using like the image- or PDF-viewer. Instead of the desktops stepping on each others toes, every desktop that identifies itself via the XDG_CURRENT_DESKTOP</code> environment variable gets its own configuration file, prefixed by the lowercased name that is read from that variable. This way different preferences on different desktops can be kept separate. This prefix will be referred to as ${prefix}</code> further down. Examples of its contents may be: mate-</code>, kde-</code>, etc. or it may be empty. Note: The XDG_CURRENT_DESKTOP</code> works completely independent of the desktop detection (in fact: the desktop detection uses it). So anyone can set this to get their own configuration space for their fresh new custom desktop. Note: XDG_CURRENT_DESKTOP</code>, like many other XDG variables is a colon separated list that is handled with a fallback cascade. i.e. if your custom xfce</code> based rice identifies itself as my-xfce-rice:xfce</code>, then the application choice will be pulled from my-xfce-rice</code> and automatically fall back to the xfce</code> defaults, even when other desktops are installed. Reading the Association Files</h4> This is a multistage search that … checks the users $XDG_CONFIG_HOME/${prefix}mimeapps.list</code></li> checks the system wide $XDG_CONFIG_DIRS/${prefix}mimeapps.list</code></li> checks again for an applications/${prefix}mimeapps.list</code> in both system and user data directories (usually ~/.local/share/</code> and /usr/share/</code>)</li> searches both user and system data directories for a prefixed with the current desktops name applications/${prefix}defaults.list</code> and applications/${prefix}mimeinfo.cache</code>.</li> falls back to searching the application files for the one that sees itself most fit for opening a file with the given mime-type using the defapp_fallback</code> function</a>.</li> </ul> The files are searched in the order of highest priority to lowest and when the first result is found it is returned. While the filenames are different those files share the same ini syntax.</figcaption> <mimtype>=<desktop-file-name>[;<another-desktop-file-name>...][;]</code></pre> </figure> For the mimeapps.list</code> files, the check_mimeapps_list</code> function searches the [Default Applications]</code> section for an entry with the given mime-type. If such an entry exists, all mentioned desktop-file names are extracted and the first one with a valid executable is returned. If not the search continues. Bug Note: Apparently there is a bug in check_mimeapps_list</code> that discards any names after the first semicolon</a>, making the loop that should handle fallback useless. The defaults.list</code> and mimeinfo.cache</code> get treated simpler than the mimeapps.list</code> files. The first result that mates the mime-type is returned, done. Sections are ignored for these files. Fallback Application Finding</h4> In case no desktop application is found using the configuration file xdg-mime</code> will start searching one itself with the defapp_fallback</code> function. This iterates over all desktop files that match the given mime-type, picking the one with the highest value in InitialPreference</code> (default is 0 when unset). In case multiple applications have the same InitialPreference</code>, the one first found or in the higher priority location wins. If a file is found this way it gets returned. Note: For the InitialPreference</code>, the desktop file specification only says that it is reserved for KDE</a>. Setting Default Applications</h2> Setting a default application for a given mime-type happens in one of two ways: For Lxqt only the make_default_lxqt</code> function gets called. For everything else there are two functions, the make_default_kde</code> one where the KDE database gets updated and make_default_generic</code> which updates the users mimeapps.list</code> file. Weirdly both are always called. (Remember the whole prefix thing?</a>) Generic Default Application Setting</h3> Surprisingly unsurprising the generic case is the simpler one here. It derives the path for $XDG_CONFIG_HOME/mimeapps.list</code> and then runs an awk script on it. This script adds or updates the <mimetype>=<desktop-file-name>;</code> association entry in the [Default Applications]</code> section that belongs to the given mime-type. What is a bit surprising is that the desktop prefix isn't handled here. For Lxqt</h3> Again qtxdg-mat</code> has a sub-command that is a prefect fit and doesn't need any wrapping: qtxdg-mat defapp --set <file> <mimetype></code></pre>For KDE 3 and 4</h3> The make_default_kde</code> function is a lot of awk code that seems to do patching on two files. Unfortunately it hasn't been updated after KDE 4</a> … KDE 4</h4> The KDE 4 specific one modifies a mimeapps.list</code> file in a directory returned by kde4-config --path xdgdata-apps 2> /dev/null | cut -d ':' -f 1</code>. It adds the mime-type to application mapping to the [Added Associations]</code> section if it is missing. My personal guess without doing further research is that this ensures that the application to mime-type mapping exists because something in KDE actually validates that an application can open a given mime-type. I'm also pretty sure that this mechanism relies on the generic setter running immediately after for setting the entry in the [Default Applications]</code> section. Maybe someone can confirm or deny this. KDE 3</h4> For KDE 3 (the default case if the version isn't 4), the profilerc</code> file in a directory returned by kde-config --path config 2> /dev/null | cut -d ':' -f 1</code> gets modified. There with ini-like syntax, a section gets added: [<mimetype> - 1] Application=<desktop-file-name> AllowAsDefault=true GenericServiceType=Application Preference=1 ServiceType=<mimetype></code></pre> Sections that act as duplicates will get removed. Installing and Uninstalling MimeType Descriptions</h2> Now that everything from getting the mime-type to the application is covered one may ask: How does the mime detection tooling know about all the file-types out there? </blockquote> On a freedesktop system this information resides in the XDG Shared MIME-info Database</a> which is a binary compiled from XML files that describe how to detect a given mime-type, and what its icon, names and descriptions (in different languages) are. xdg-mime</code> makes it easier to install those XML descriptions. (When packaging follow the target distributions guidelines first!) For the general case installing puts/deletes the given file in the desired mime/packages</code> directory. For user mode that is $XDG_DATA_HOME/mime/packages/</code>, defaulting to ~/.local/share/mime/packages/</code>. For the system mode $XDG_DATA_DIRS/mime/packages/</code>, defaulting to /usr/share/mime/packages/</code> and /usr/local/share/mime/packages/</code>. There is also a fallback for a KDE3 mechanism called "mimelink" but I will skip that one for now. Note: Don't confuse the files in the packages</code> directory with other XML files named like the mime-types themselves. The ones in packages</code> really are packages containing the descriptions for multiple mime-types while the other ones are already unpacked for better discovery. Hooks on the Mime Database</h3> After (un)installing the update_mime_database</code> function takes care of making sure that the mime database and caches get rebuilt. When in user mode, the currently running desktop is KDE and a display is present kbuildsycoca</code>, kbuildsycoca4</code> or kbuildsycoca5</code> is called, which rebuilds KDE configuration caches. After that every directory in $PATH</code> with the addition of /opt/gnome/bin</code> is searched for a update-mime-database</code> program. All found binaries are ran with the mime database folder that was just updated as the first argument. Note: update-mime-database</code> is the official program for rebuilding the mime database. Relevant Specifications</h2> The XDG Shared MIME-info Database</a> which allows storing information about mime-types themselves.</li> The XDG MIME Applications Associations</a> which are responsible for mapping applications to mime-types.</li> </ul> Until Next Time …</h2> On expectations: When starting to write this article I didn't expect it getting this long and technical, but researching as well as finding and fixing bugs ended up producing a lot of information which I didn't want to hide. A huge thank you goes to Kevin Krammer and Jeremy White for starting the xdg-utils</code> project, to Simon Lees who is currently the de-facto maintainer and everyone who contributed in any way. And while the xdg-utils</code> might be a mess, they are a bloody awesome and quite functional mess! Recently xdg-utils</code> has received a beta release</a> after a very long time. You can help the project by testing it, reporting and researching issues, giving context and writing fixes or documentation. Next up on this mini-series will probably be a dive into xdg-settings</code>, but please don't quote me on that. "It just opens files" - xdg-open under the hood 2023-09-02T00:00:00+00:00 What is xdg-open ?</h2> xdg-open</code></a>, part of the xdg-utils</a> package is a freedesktop.org</a> tool that aims to give you (and everyone else) a desktop independent interface for opening a file or URL, taking care of all the environments quirks and settings so the application or script that "just wants to open a file" doesn't have to. Examples for opening a file, a URI and a file://</code> URI with xdg-open</code></figcaption> xdg-open example.txt xdg-open https://slatecave.net xdg-open file:///usr/bin/xdg-open</code></pre> </figure> This article is based on version 1.1.3</code>, the version that is currently in development is called 1.1.3+</code>. Questions</h2> Before starting out I was trying to find answers to the following questions: How does xdg-open</code> know my favourite applications?</li> Which ways are there to configure it?</li> Is there an interface to extending it?</li> </ul> This is why this article is about high level logic rather than little tricks. First look at the file</h2> To analyse any tool one should try to get it's source code. A quick command -v xdg-open</code> reveals the binary to be under /usr/bin/xdg-open</code> for me, file /usr/bin/xdg-open</code> tells me that it looks like a POSIX</a> shell script</q> and a shellcheck /usr/bin/xdg-open</code> command reveals that file</code></a> was not quite right about the POSIX part. But it is a shellscript that anyone can just open in their favourite text file viewer/editor. The file contains the following: UI code to make it easier to use</li> Helpers for querying .desktop</code> files</li> Lots of checks for a lot of things</li> cli parser code</li> Heuristics for detecting desktop environments</li> Workarounds for tools that seem to misbehave</li> Wrappers for various desktop specific openers</li> Heuristics spaghetti with lots of "xdg" and "generic" in them (I assume those are the really interesting parts)</li> One giant switch that decides what to do</li> </ul> In case you don't have a local copy of a recent xdg-open: Its source lives in the freedesktop gitlab in xdg-open.in</a> and xdg-utils-common.in</a> which is reused across the entire xdg-urils</code> package. This article will link to the 1.1.3</code> tag instead of the latest version. What does it do?</h2> xdg-open</code> really only does one thing, open whatever one throws at it and therefore has a pretty linear flow. It only has options to query help and version information, I'll ignore those code parts. Note: Those options are checked for near the start of the script so a xdg-open --help</code> doesn't even have to load all the logic. Note: There is an undocumented XDG_UTILS_DEBUG_LEVEL</code> environment variable that makes xdg-open</code> more chatty when set to 1</code>. get one URL or filename from the cli arguments</li> try to find out which desktop is currently in use using detectDE</code>.</a></li> fall back to a generic</code> desktop if detection wasn't successful.</li> remove itself from the BROWSER</code> environment variable</li> based on the detected desktop call the appropriate helper for opening the file.</a></li> </ul> Desktops</h2> The "desktops" xdg-open</code> is aware of are (in order of them appearing): kde</code> </dt> Both modern day and very old KDE</a> desktops. </dd> deepin</code> </dt> The Deepin Desktop Environment</a> </dd> gnome3</code> </dt> The GNOME desktop</a> in versions 3 (and 4) </dd> cinnamon</code> </dt> Cinnamon</a>, the Linux Mint</a> Desktop, it is treated as if it was gnome3</code>. </dd> gnome</code> </dt> The GNOME desktop in version 2 </dd> mate</code> </dt> The MATE Desktop Environment</a> that continues as a modern version of GNOME 2, but gets treated differently. </dd> xfce</code> </dt> The Xfce Desktop Environment</a>. </dd> lxde</code> </dt> The Lightweight X Desktop Environment</a>. </dd> lxqt</code> </dt> The Lightweight Qt Desktop Environment</a>, a qt remake of Lxde, but not related enough to share code here. </dd> enlightenment</code> </dt> The Enlightenment</a> Desktop. </dd> </dl> Some of them aren't even desktops: cygwin</code> </dt> We are running under Windows in a Cygwin</a> environment. </dd> wsl</code> </dt> We are running in a WSL environment. Note that this is not yet present in version 1.1.3</code>. </dd> darwin</code> </dt> We are running on MacOS. </dd> flatpak</code> </dt> We are in a flatpak</a> container. </dd> generic</code> </dt> This is a desktop that doesn't require any special treatment 🥳 </dd> </dl> Good news: many of those seem to map to the open_generic</code> function, but for some desktops assumptions are made. Update note regarding Deepin: Previously this article listed deepin</code> as dde</code>, which is icorrect. The XDG-utils use the string deepin</code>. Detecting the desktop</h3> Desktop detection is done by the detectDE</code> function</a> using multiple heuristics in order of decreasing accuracy. The result is written to the DE</code> global script variable. The following tests are executed in order given that all previous tests didn't match: Test if the XDG_CURRENT_DESKTOP</code> environment variable contains a known value.</li> Test for desktop specific clues KDE_FULL_SESSION</code> is set</li> GNOME_DESKTOP_SESSION_ID</code> is set</li> MATE_DESKTOP_SESSION_ID</code> is set</li> The org.gnome.SessionManager</code> name is owned by some application on the d-bus</li> The _DT_SAVE_MODE</code> xprop on the root window contains the string xfce4</code></li> xprop -root</code> has xfce_desktop_window</code> at the start of a line in its output</li> The DESKTOP</code> environment variable begins with Enlightenment</code></li> LXQT_SESSION_CONFIG</code> is set</li> </ul> </li> Test if the DESKTOP_SESSION</code> environment variable contains a known value.</li> Test if uname</code> outputs either CYGWIN</code>, Darwin</code> or Linux</code> In case of Linux, if /proc/version</code> contains the string microsoft</code> and the command powershell.exe</code> is present, assume wsl</code>. (This was added very recently</a> and is not in version 1.1.3</code>)</li> </ul> </li> </ol> The following tests always run: When the desktop is a gnome</code>, test if the gnome-default-applications-properties</code> command is present, if not, assume gnome3</code></li> Test if $XDG_RUNTIME_DIR/flatpak-info</code> is present and if yes assume flatpak</code> (Why isn't this the first check?)</li> </ol> Note: Because this is implemented in xdg-utils-common.in</code> this also applies to all of the other xdg-utils tools. Desktop specific openers</h3> Since xdg-open</code> tries to offload the work when the desktop environment has its own mechanism for opening files lets cover those first. Desktops that "just call an opener"</h4> Desktop</th> Opener</th> Fall back to open_generic</code></th></tr></thead> deepin</td> dde-open</td> yes</td></tr> enlightenment</td> enlightenment_open</td> yes</td></tr> lxqt</td> </td> always</td></tr> cygwin</td> cygstart</td> no</td></tr> darwin</td> open</td> no</td></tr> wsl</td> powershell.exe start</td> no</td></tr> </tbody></table> flatpak</h4> When in a flatpak xdg-open</code> invokes the Desktop</code> portals OpenURI</code> function using gdbus</code></a>. gdbus call --session \ --dest org.freedesktop.portal.Desktop \ --object-path /org/freedesktop/portal/desktop \ --method org.freedesktop.portal.OpenURI.OpenURI \ "" "$1" {}</code></pre> Note: for the 1.1.3+</code> version xdg-open</code> uses the org.freedesktop.portal.OpenURI.OpenFile</code> method for files and file URIs. It also added a timeout of 5 seconds. GNOME, MATE and Xfce</h4> GNOME 2, GNOME 3, MATE and Xfce are related in that they all share a similar cascade of openers (falling back to the next one, if the previous one isn't present) is used. exo-open</code> when on xfce</li> gio open</code></li> gvfs-open</code></li> gnome-open</code> when on gnome</code> (GNOME 2)</li> mate-open</code> when on mate</code></li> open_generic</code></li> </ul> Note: for GNOME 3 the only options really are gio open</code> and gvfs-open</code> before falling back to the generic opener. KDE</h4> For KDE, if the KDE_SESSION_VERSION</code> environment variable is set kde-open</code> is used for KDE 4 and kde-open5</code> for for KDE 5. Otherwise kfmclient exec</code> (with some logic to fix the exit code) is used for even older KDE versions. KDE 6 will use kde-open</code> again</a>, but that isn't in version 1.1.3</code>. Lxde</h4> For Lxde pcmanfm</code> is used if present, but only for filepaths and file://</code> URIs. Otherwise falling back to open_generic</code>. Generic Opener</h2> The generic opener is invoked by the calling the open_generic</code> function</a> with the filepath or URI as the only argument. This is yet another heuristics spaghetti function that tries to delegate work to other tools that may already be configured. Opening files</h3> In case the passed argument looks like a filepath or file://</code> URI, xdg-open</code> tries some additional openers. If the file doesn't exist the script will exit with an appropriate error message. If an X or Wayland display is present (checked using has_display</code>) the files mime-type is determined using xdg-mime query filetype</code> and the function open_generic_xdg_mime</code></a> is attempted for opening. If no display is present or open_generic_xdg_mime</code> returned, opening with run-mailcap</code></a> is attempted if present. run-mailcap</code> apparently is the Debian solution to the problem from before freedesktop standardised file opening. If we have a display and mimeopen</code></a> from the File-MimeInfo perl package</a> is present attempt to open using that. If all of those miss then the open_generic</code> function continues with treating the first argument given to it as if it was an URI. Opening URIs</h3> For URIs the following happens until one of the steps finds a way to open the URI. If there is a display (X or Wayland, checked by has_display</code>) try to derive a URI scheme and open it using open_generic_xdg_mime</code></a> with a mime-type of x-scheme-handler/<scheme></code>. This allows registering custom URI handlers using the already existing mechanism for mime-types. If the BROWSER</code> environment variable (a colon :</code> separated command list) is set try all browser commands in that variable (if %s</code> is present insert the URI in the desired position using printf</code>) until one of them succeeds. (implemented in open_envvar</code></a>) If unset try a set of sane defaults depending on whether a display is present or not. Note: The BROWSER</code> variable was cleaned up at the start of xdg-open</code>. At the time of writing the list of browser commands was … … when a display is present: x-www-browser</li> firefox</li> iceweasel</li> seamonkey</li> mozilla</li> epiphany</li> konqueror</li> chromium</li> chromium-browser</li> google-chrome</li> </ul> … for the terminal (always attempted): www-browser</li> links2</li> elinks</li> links</li> lynx</li> w3m</li> </ul> open_generic_xdg_mime</code></h3> This function</a> takes a filepath (or URI) and a mime-type as input. It uses xdg-mime query default</code> to get the name of the .desktop</code> file, then searches for that in all $XDG_DATA_DIRS/applications/</code>, when found parses the Exec</code> and other needed fields, runs the resulting command and makes the script exit. If it didn't find a matching file it returns. Summary</h2> xdg-open</code> tries to open a file or URI. If it recognises the environment and knows a opener that is specific to that environment</a> it will use that. If there is no specific opener it will fall back to a cascade of generic openers, preferring the freedesktop way, falling back to legacy openers. The freedesktop mime-type to application mapping uses xdg-mime</code></a>. For URIs it will attempt to open the URI as if it was a file with the mime-type x-scheme-handler/<scheme></code>. The last configurable fallback is the BROWSER</code> environment variable which is a colon :</code> separated list of commands of which all will be tried until the first one succeeds. If %s</code> is present in the command the URI or filepath will be inserted in its place instead of being appended. If the BROWSER</code> environment variable is empty, it falls back to a list of hard coded default applications. Extending</h3> For fully custom behaviour wrapping xdg-open</code> still seems to be the best choice.</li> Defaults can be customised by setting a handler in the BROWSER</code> environment variable.</li> xdg-open</code> listens to what xdg-mime</code> says on which mime-type a file is and which application to use for opening.</li> </ul> Expectations</h2> That was quite a bit more desktop-specific stuff in there than expected, while the generic part was supisingly simple at the high level logic view. One thing that was unexpected was that there is no obvious connection to xdg-settings</code></a> i.e. for the default web browser, but maybe that is hidden in xdg-mime</code>. I'll try to follow up with articles on both tools. Make sure you subscribe to my atom feed</a> 😉. Bye! Improving an fzf example with null bytes 2023-07-09T00:00:00+00:00 What is the problem?</h2> On the fzf manpage</a> the --preview-window</code> option is shown off using a combination of git grep</code> for querying every line of every file in the current git repository, fzf</code> for searching and bat</code></a> for a preview with the line currently selected in fzf highlighted and scrolled into the middle of the preview window. (pretty awesome example if you asked me.) The example from the manpage</figcaption> # Preview with bat, matching line in the middle of the window below # the fixed header of the top 3 lines # # […] git grep --line-number '' | fzf --delimiter : \ --preview 'bat --style=full --color=always --highlight-line {2} {1}' \ --preview-window '~3,+{2}+3/2'</code></pre> </figure> The example works by relying on the fact that the output of the git grep</code> looks like <filename>:<linenumber: …</code>, split at the colon, {1}</code> is the filename, {2}</code> is the line number. While the example is very good at explaining how the --preview-window</code> interacts with the --delimiter</code> (its primary purpose), it breaks once there is a colon somewhere in a filename, then {1}</code> and {2}</code> become two halves of an incomplete filename and bat has no idea what it should do with that. Example output produced by git grep --line-number ""</code>. Note that the last line looks a bit funny.</figcaption> test.txt:1:Lorem ipsum test.txt:2:dolor sid … throw:off:1:Does this filename confuse fzf? </samp></pre> </figure> Let's fix it!</h2> The Solution is actually pretty simple: git grep</code></a> supports a -z</code> flag, making it use null bytes instead of colons for separating filename, line number and content, tell fzf to use that as the delimiter using --delimiter '\0'</code> and, the preview isn't confused anymore. One remaining problem is that the menu looks like this: test.txt1Lorem ipsum</samp>, not very readable because those nullbytes are invisible. But there is nothing stopping us from replacing all null bytes with nullbytes followed by a colon and telling fzf to use that combination as a delimiter. git</code> doesn't have something built in for that, but that is what sed</code> was built for. The example from the manpage modified to use null separators in addition to human readable ones.</figcaption> git grep -z --line-number "" | sed -E 's|\x00|\x00:|g' | fzf --delimiter '\0:' \ --preview 'bat --style=full --color=always --highlight-line {2} {1}' \ --preview-window '~3,+{2}+3/2'</code></pre> </figure> In case this looks a bit dull add a --color=auto</code> to the git command and a --ansi</code> to fzf. Done</h2> Given that the more correct version takes noticeably more brain-cycles to decode it is no surprise they used the simple version for the manual. In case you want to: Learn more about null separation in the shell</a>. Happy experimenting! Slatectx: Adding some context to the desktop 2023-04-23T00:00:00+00:00 Why I created slatectx</h2> To put my usecase and workflow in a few sentences: What I'm mostly doing is editing textfiles, writing code, configuration, notes and so on. I'm doing a lot of thing in the shell (bash) and my text editor (kakoune</a>), usually working with more than one window open using i3 or sway as a window manager. Leading to … I want a new terminal and I want it in the same pwd as the last terminal I interacted with on that workspace! </blockquote> The workspace constraint is because I sometimes switch between a reference or otherwise related project to the main thing I'm working on and I don't want the pwd on one to influence the pwd on the other and usually I already separate those by putting them onto a different workspace. Identifying Contexts</h2> To be able to do anything that is context aware, one needs a way to identify a certain context and a way of finding out which context currently applies. Since this journey started with a simple POC and then stayed pretty simple slatectx contexts are named by opaque (no assumptions based on the name) strings containing alphanumeric characters, dashes -</code> and underscores _</code> with slashes \</code> being forbidden because these have to work in filenames. Generating Context Names</h3> With the requirements being that I don't want an additional daemon for doing housekeeping and be able to debug where the names are coming from I settled on the following rough naming scheme: <session_type><sesion_id_or_display>-<workspace></code></pre> For example, the workspace I'm writing this post on maps to the context y1-9</code>, derived from me currently using a wayland session that has wayland-1</code> set as its WAYLAND_DISPLAY</code> and being on workspace 1. With an ssh connection this would simply be ssh</code> and tmux uses t-<tmux_pid></code>. To generate those identifiers my approach involved finding out which session type is currently used</a>, getting the workspace</a> and then stitching that together in the slatectx-get-context-name</code></a> which is the central interface for getting the name of the current context (A bit like xdg-open for opening files). The slatectx-get-context-name</code> script is supposed to be replaced when one wishes to use a different context naming logic. Switching directories</h2> With the problem of figuring out the current context out solved one can implement things like a mot recently used directory (the original motivation). The logic behind that has now moved to its own tool slatectx-pwd</code></a>. When used with the set command it validates that the directory exists (and is a directory), turns a relative into an absolute path and stores the result in $XDG_RUNTIME_DIR/slatectx-pwd/$(slatectx-get-context-name)</code>. The get operation reads this file, runs the checks again (the filesystem could have been modified in between) and prints the path out, depending on the check results with an error return code. This little helper is used in my bashrc</a> to cd</code> into the last used directory on startup and hooked into the PROMPT_COMMAND</code> to record the last used directory. A similar thing happens with a filedialog script</a> I'm using to quickly navigate the filesystem and open files, which on startup gets passed the result of slatectx-pwd get</code> and has a hook to set the pwd on a change-directory event. Another thing that is set up in my shell and text-editor is a cdx</code> alias for quickly jumping to the slatectx-pwd in an already open session. There is also experimental configuration for integrating it with lf</a> which I don't use as often as I'd like to, because I haven't figured out how to make use of lf in a way that makes me not run into a wall at some point where a command I expect isn't there because I haven't figured out how to configure that part of lf yet. (one of the reasons I wrote the filedialog script) Shared text-editor</h2> One thing I missed after switching to kakoune from gedit was that with gedit I could just have a keybind that opened another text editing window and I could just move the tabs between them (not missing the tab UI though). With kakoune having a client-server architecture one could replicate having multiple clients share the same open buffers, if only there was a way to find out what the server is call … yes one can tell kakoune what the socket should be called and whether it should start in client or server mode. That's what the kak-slatectx</code></a> and the kak-e</code></a> are for. kak-slatectx</h3> kak-slatectx</code> replaces the kak</code> command (using an alias) and by default tells kakoune to start with the given slatectx context as the socket name in client or server mode depending on the presence of the socket file. To make it a full replacement, this behaviour is completely disabled, should one attempt to pass the -c</code> or -s</code> options to kakoune. kak-e</h3> kak-e</code> is the slatectx enabled fileopener for kakoune, for it to work kakoune has to remember the last window one interacted with, which is a pretty simple FocusIn</code> hook paired with a custom current_client</code> option</a>. kak-e</code> constructs an edit command for kakoune that then opens the given files in the last focused window, launching a new kakoune instance if needed. Reusing an image viewer</h2> Another thing I set up a wrapper for is imv</a> (which at the time of writing is looking for a new maintainer!), an image viewer which also has a socket for remote controlling it. Since imv always uses its pid for this, an approach similar to slatectx-pwd is used to store the pid of the currently running imv instance and then using that like kak-e</code> to open files in an already open window (Which currently is a bit glitchy). imv-slatectx</code></a> also has an option to disable launching an imv instance, which is useful for preview displays. Open an Image, look at it select the next and the image viewer automatically jumps to the selected image. Close the image viewer and the previews are gone with more screen real estate for managing files. (I've integrated it like this into my lf configuration and it is pretty useful) Conclusion</h2> Here I described why and how I used some relatively simple shellscript to implement some shared pieces of information that can be used to make applications work together. Something I miss with most GUI applications that maybe have a hared recently used, but completely ignore what the one using them is currently working on. That said: Feel free to to reuse my code and ideas on this one and nag me on writing documentation and packaging if you are interested. What is NAT and why you don't want it 2023-03-27T00:00:00+00:00 You may or may not have heard of NAT, Network Address Translation, here is how it works, why you don't want it and why you probably have to use it anyway. How the Internet was supposed to work</h2> The original idea of the internet was that every machine is reachable, can directly connect to other machines using nothing but their public address and also offer serveries. Which worked pretty well until … Why the Internet community started NATing</h3> … the Internet started running out of publicly reachable addresses. So everyone started assigning private IP-Addresses to their internal networks, trying to get the most out of the precious public address space. Legacy NAT</h2> Legacy NAT is simply swapping one IP-Address for another, with a 1:1 relationship between private and public addresses. Either statically assigned or on demand to a machine that wants to connect, so if you wanted to connect to the Internet you have to wait for a free Line (public Address) from a pool of available addresses (called a NAT Pool). Hello old dialup! These are respectively called Static and Dynamic NAT. Modern NAT - Port Address Translation (PAT)</h2> Nowadays with 10s of devices per household with active connections to outside servers that Legacy approach of give a machine an IP when it needs one and recycle after doesn't work anymore. But one can abuse the fact that most machines don't offer services for the public and therefore have a lot of outgoing and no incoming connections. So instead of assigning whole IP-Addresses we made our Routers translate combinations of Layer 4 ports and IP-Address. If one now wants to connect to the server at 1.2.3.4</code> on port 443 from 192.168.0.10</code> from port 12345 via a router assigned 203.0.113.3</code> (it would with multiple public addresses, too) as its public address … The router tries to find a port on its public address using either the original source port or if already in use because someone else behind the NAT had the same idea of a great source port a port near the original port. Let's say we ended up with 12347.</li> Now the router replaces the internal address with the public one and swaps out the port for the publicly reachable one and remembers the combination until the connection closes, there is a timeout.</li> If now an answer arrives on 203.0.113.3</code> port 12345 the NAT knows that the real destination is 192.168.0.10</code> on port 12345 and vice versa.</li> </ul> For ICMP the identifiers can be used similar to port numbers. UDP based protocols need additional support from the NAT as UDP itself has no way of matching requests with replies. With that setup one can support up to ~60000 concurrent TCP connections with a single public IP-Address, great isn't it? Carrier grade NAT</h2> Well, your provider most probably thinks that that is great because now they can serve a whole lot of customers and still only pay for a handful of public IPs. There is even the 100.64.0.0/10</code> network reserved in RFC 6598</a> for exactly this purpose. Problems with NAT</h2> The problem with this is now that your average client is behind two or even more layers of NAT, one at their provider, one built into their home-router. That comes with at least two problems: For real-time applications (Videochat,SIP) you want as direct of a connection as possible which is the opposite of what a NAT does.</li> Long running connections (i.e. your chat or Mail-client waiting for realtime notifications or an ssh session) may get terminated because the NAT needs free ports because 60000 connections is surprisingly little scaling up.</li> </ul> Also NAT actively hinders innovation. Ever heared of SCTP, the Stream Control Transmission Protocol</a> (RFC 9260</a>)? One of the reasons you have probably never seen it in the wild is because old NATs (like Joe Averages router running outdated firmware) have no clue what to do with it. Same problem with QUIC</a> (RFC 9000</a>) which is based on UDP, because punching a hole into the NAT that has the right shape is usually difficult to impossible you probably won't find it in the wild. NAT64</h2> NAT64 was a way to translate IPv6 adresses in the 64:ff9b::/96</code> network to IPv4 but it has been deprecated in favour of dualstack configurations. Probably because for that to work either direct application support is needed or a dns Server which has to translate IPv4 addresses to the NATable IPv6 ones. Myths around NAT</h2> Great I don't need a Firewall!</h3> Yes you do! A NAT only denies incoming connections because no machine in the internal network currently punched a hole in it with UPnP. Your NAT is also waaay too complex for trusting it with denying unwanted connections. (Don't confuse it with a NAT built into a firewall) I want IPv6 NAT because of Privacy!</h3> If you have this kind of network access you can as well reassign random addresses using your favourite IPv6 address assignment mechanism. Solving the NAT problems</h2> So you don't like your NAT anymore? What if the Internet had an address space large enough for every device? Well its your lucky day the solution is called IPv6 and if in 2023 your ISP doesn't give you IPv6, ask them nicely why they are stuck in the past. Building an animated element 2023-03-17T00:00:00+00:00 Accordion elements on the web seem to be everywhere and almost everyone seems to implement them in a way that only works for the well sighted user with a modern browser with JavaScript enabled. I want to show that these can be built in a better way. So why are accordions used at all?</h2> While using accordions or expanders in some situations is a bad idea, sometimes they help with uncluttering, hiding information that isn't needed by most people but available from the already loaded document on request when needed. An example would be to provide contact details on a page, usually visitors don't care, but when they do, that information is easy to find. Another example would be an event log with the summaries always being visible and the details being available inline on request. sourcehut and YunoHost use this pattern. Note: Accordions usually are multiple collapsible elements grouped together and only allow one open at a time. I'm not going into that functionality here. The HTML way of building an accordion …</h2> This pattern is so useful that HTML has elements for exactly that usecase, the <details></code> and <summary></code> elements, one simply puts a summary inside a details tag along with some content and it works, in all browsers, without JavaScript, with any input method the browser supports. Note: Apparently the details element and accessibility is it's own can of worms, too</a>. As the article mentions: You decide. Example of such a details element</figcaption> <details> <summary>Your DNS records for example.org are not configured</summary> <p>Add an <code>A</code> record pointing to <code>93.184.216.34</code>.</p> <p>Add an <code>AAAA</code> record pointin to <code>2606:2800:220:1:248:1893:25c8:1946</code></p> <p>It is important to set both, otherwise peopley may be unable to access your server.</p> </details></code></pre> It then would be displayed similar to this: Your DNS records for example.org are not configured</summary> Add an A</code> record pointing to 93.184.216.34</code>. Add an AAAA</code> record pointing to 2606:2800:220:1:248:1893:25c8:1946</code> It is important to set both, otherwise people may be unable to access your server. </details> </figure> What people are doing instead …</h2> You may have been surprised by the fact that there is an HTML element for exactly this purpose because you have never seen one in the wild. There are probably three reasons people don't use it: They simply don't know that it exists</li> They found out, tried to animate it and failed to do so</li> They use a library which fell victim to one of the previous reasons.</li> </ul> The result being that they give up and write their own accordion from scratch, button onclick</code> handler, element with changing styles, done. Except that the result is usually an accessibility nightmare, and completely falls apart when for some reason the JavaScript needed to operate it isn't available. Why is the details element seemingly hard to animate?</h2> Animating a details tag is simple, one can use the [open=""]</code> CSS selector to find out whether the details element is currently open, put some keyframes and an animation (more details later) on it and it opens with a smooth animation and closing is … not animated at all. No your CSS isn't at fault here. How the details element works.</h3> The details element is a bit of a unicorn in that it manipulates the page tree. When the content gets shown elements are added and when it closes the elements get removed from the page instantly, and while that is great as it saves resources PR is not happy because they want an animation. My personal guess is that this is where most developers who know about the details element give up. Animating it Anyway</h2> So we want an animation for sighted people with modern browsers and a good experience for everyone else, so lets try to achieve that by using the web as it was intended, HTML for content, CSS for eye candy and JavaScript for providing some optional functionality. Note: Animating slightly degrades accessibility (which is probably still better than with your completely custom accordion) of the details element. You can find the final result over on Codeberg.</a> Let's start with the content, I'll assume you have a plain details and summary tag filled with your content and we want to leave it as it is. <details> <summary>Your DNS records for example.org are not configured</summary> <p>Add an <code>A</code> record pointing to <code>93.184.216.34</code>.</p> <p>Add an <code>AAAA</code> record pointin to <code>2606:2800:220:1:248:1893:25c8:1946</code></p> <p>It is important to set both, otherwise peopley may be unable to access your server.</p> </details></code></pre> In case you want to follow along but only have a desktop browser: open a new tab, navigate to about:blank</code> and open up the developer tools. Depending on the browser you now have a pretty good IDE with live preview. The Grand Opening</h3> Before we worry about closing the element, lets worry about it opening. The simplest way here is to simply animate the max-height</code> property from the current height of the element (--details-current-height</code>) to one screen height (100vh</code>), filling only back in time in case the details element is taller than one screen height (i.e. on a smaller device), in that case it will simply snap to full height off screen and is fully viewable. Because we currently have no idea what the current height of the details element is, falling back to an assumed 1em</code> as the height of our summary element works pretty well. (You may have to adjust this based on extra padding and margin you are applying) @keyframes details-appear { from { max-height: var(--details-current-height, 1em); } to { max-height: 100vh; } } details[open=""] { animation: 1s linear details-appear; animation-fill-mode: backwards; /needed for the max-height animation to work/ overflow: hidden; }</code></pre> With that the opening part is taken care of. Fading out of Existence</h3> Because the details element immediately closes and there is no way to animate that we have to put a delay between the actual interaction with the element and the open</code> attribute being removed, so let's add some JavaScript. Because we want to hook on the interaction of a sighted user with the summary element, we use an onclick</code> handler here and assume a details_click_handler()</code> function. The following function simply adds the onclick</code> handler to every summary element on the page (if you only want to target specific elements … I'll leave that as an exercise). function initalize_animated_details() { var details_elements = document.getElementsByTagName("summary"); for ( let i = 0; i < details_elements.length; i++) { details_elements[i].onclick = details_click_handler; } }</code></pre> To run this initalizer you can use any method you like, but since in my experience document.onload</code> is a bit wonky when testing inline code on small documents here is a way to defer it until the first interaction. document.onclick = (event) => { / we only need it once / document.onclick = undefined; initalize_animated_details(); / if the event was for a summary tag run the handler / if (event.target.tagName === "SUMMARY") { return details_click_handler(event); } / Otherwise let the browser handle the click / return true; }</code></pre> With that set up we can now focus on the click handler, which has to do two things: add a style-class that will trigger the animation</li> set a timer to actually close the element after the animation.</li> </ul> A bare minimum implementation of the details_click_handler</code></figcaption> function details_click_handler(event) { let details = event.target.parentElement; / When opening, let the browser do its job / if (!details.open) { return true; } details.classList.add("closing"); setTimeout(function() { details.open = false; details.classList.remove("closing"); }, 500); return false; }</code></pre> Fun fact: if you try that one out right now you'll get the experience of someone who is unable to see the closing animation.</figcaption> </figure> With the bare minimum of JavaScript added let's add some CSS to get a proof of concept animated details element. This is basically the reversed opening from the current height, defaulting to 100vh</code> to the height of the summary element defaulting to 1em</code> again. Again we'll worry about actually setting those variables to something useful later. @keyframes details-disappear { from { max-height: var(--details-current-height, 100vh); } to { max-height: var(--details-summary-height, 1em); } } details.closing { animation: var(--details-close-animation-length, .5s) linear details-disappear; overflow: hidden; max-height: var(--details-summary-height, 1em); }</code></pre> Now you should have a not pretty but animated closing details element. Improving the Animation</h2> Now that we have a proof of concept, lets make it usable. The issues we currently have are: Handle an interaction while the animation is playing correctly.</li> Accessibility impact is larger than it could be.</li> The heights used are still placeholders.</li> The animation speeds don't match up.</li> </ul> Handling an interaction while the animation is playing</h3> Currently when one interacts while the animation is playing the behaviours is that the element keeps closing, which is not great. To fix it we have to add some additional checks to our handler function to test if the closing</code> class is present on an open element. If that's the case we just remove the class gain instead of launching a new timeout and on the timeout part we just discard the timeout if the closing class isn't present. Lowering the impact on accessibility</h3> There are two problems I'm trying to solve with this one: When only relying on something that uses the accessibility interface of the browser (i.e. screenreader only) there is an inexplicable delay between interaction and the element being announced as collapsed, very confusing and/or annoying.</li> When navigating by keyboard, no matter what the output device is, the content inside the details element can still be focused while the animation is playing, again potentially confusing and/or annoying.</li> </ul> Each of the alone would be enough of a reason (at least for me) to not animate the closing of the element on keyboard interaction. (Fixing this is probably possible but at that point we are writing a full blown expander in JavaScript.) Based on this we make a simple assumption: If you are not using a pointing device, you are probably not in the group that greatly benefits from an animation. Note: Of course the "uses a pointer" is a heuristic that will be sometimes be wrong. To achieve this one can test for the offsetX</code> and offsetY</code> values of the event to find out weather the event came from a pointer or not, both are 0 they probably didn't come from a pointer and we tell the browser to do what it would do without us interfering. To stop the opening animation from playing when we won't deliver a corresponding closing animation we add an animated</code> class as an additional requirement in our two animation CSS rules. The offset check to only play animations for events from pointer devices</figcaption> function details_click_handler(event) { let details = event.target.parentElement; if (event.offsetX == 0 && event.offsetY == 0) { details.classList.remove("animated"); return true; } details.classList.add("animated"); // … Rest of the function goes here … }</code></pre> </figure> This way we can make PR and our bosses happy while mostly staying out of the way of people who appreciate clean websites more than fancy animations. Note that after this the animation won't play if the JavaScript isn't running (The element will just snap open and closed like at the start). Deglitching the animation</h3> Now that we have taken care of accessibility issues, we can put the CSS variables introduced earlier to use to make the animation responsive to the current state to avoid some glitches. This means populating the --details-summary-height</code> and --details-current-height</code> properties directly on the details elements. This code is best placed after the offset check so that the properties are already set when opening and closing</figcaption> let summary_height = event.target.getBoundingClientRect().height; let current_height = details.getBoundingClientRect().height; details.style.setProperty('--details-summary-height', summary_height+"px"); details.style.setProperty('--details-current-height', current_height+"px");</code></pre> </figure> Getting the timings to match</h3> After that is fixed the last open issue is, that the speed of the animation opening and the speed of the animation closing don't match. With the opening speed currently being one screen height minus summary per second it is pretty easy to get the closing animation to match that as when the accordion is open we can just ask the browser how tall the details and summary elements are, how tall the screen is and calculate the animation length from that. Since we already know the heights of both relevant elements we'll use the variables from earlier. This code calculating and setting the animation length should be put right before we add the closing</code> CSS class</figcaption> let close_animation_duration = (current_height-summary_height)/window.innerHeight; details.style.setProperty('--details-close-animation-length', close_animation_duration+"s");</code></pre> </figure> To make the actual state match the timeout previously set to 500</code> should now be close_animation_duration1000+10</code> the extra 10ms is to allow the browser a bit of breathing room so that we don't cut the animation off before it has finished. The final result</h3> The resulting details_click_handler</code> function now should look like the following. function details_click_handler(event) { let details = event.target.parentElement; // if called for a non-pointer event don't play animations if (event.offsetX == 0 && event.offsetY == 0) { details.classList.remove("animated"); return true; } details.classList.add("animated"); // tell the animations where to start and where to end let summary_height = event.target.getBoundingClientRect().height; let current_height = details.getBoundingClientRect().height; details.style.setProperty('--details-summary-height', summary_height+"px"); details.style.setProperty('--details-current-height', current_height+"px"); // log the event for debugging // console.log("click", event); // let the browser handle the opening if (!details.open) { return true; } if (details.classList.contains("closing")) { // abort a closing animatiton when interrupted details.classList.remove("closing"); } else { // calculate close animation length let close_animation_duration = (current_height-summary_height)/window.innerHeight; details.style.setProperty('--details-close-animation-length', close_animation_duration+"s"); // trigger closing animation and correponding timeout details.classList.add("closing"); setTimeout(function() { // discard timeout if someone interrupted the animation if ( details.classList.contains("closing")) { // properly close the element and clean up the styleclass details.open = false; details.classList.remove("closing"); } }, close_animation_duration1000+10); } return false; }</code></pre> And our final CSS: @keyframes details-appear { from { max-height: var(--details-current-height, 1em); } to { max-height: 100vh; } } @keyframes details-disappear { from { max-height: var(--details-current-height, 100vh); } to { max-height: var(--details-summary-height, 1em); } } details.animated[open=""] { animation: 1s linear details-appear; animation-fill-mode: backwards; overflow: hidden; } details.animated.closing { animation: var(--details-close-animation-length, .5s) linear details-disappear; overflow: hidden; max-height: var(--details-summary-height, 1em); }</code></pre>Updates</h2> 2023-03-25</h3> Added note about accordions usually having a grouping functionality.</li> Added link to a [great article by Scott O'Hara explaining the problems around how details element is implemented in browsers].</li> Added explanation why I'm disabling the animation for non-pointer inputs here.</li> </ul> Having a look at greetd 2022-08-16T00:00:00+00:00 What is greetd?</h2> greetd is a minimal and flexible login manager daemon that makes no assumptions about what you want to launch. </blockquote> In other words Kenny Levinsen</a> created an awesome piece of software that allows anyone to easily launch anything on login and that without being bound to a fronted. You can find greetd on sourcehut </a> Most information in this article came from reading the greetd code and making notes on good old paper. Note: Kenny Levinsen also wrote an interesting blogpost on login managers.</a> Note: If you want to understand how greetd works please read the code yourself, this article alone is not enough! (but hopefully helpful) In case you find a mistake or have feedback please contact me.</a> How greetd is structured</h2> Greetd has a pretty modular approach to things and is made of a bunch of components one can look at separately. The managing greetd server daemon (main focus)</li> The greetd --session-worker</code> process (good to know)</li> The greeter</li> </ul> I won't be looking at the greeter as the interface is pretty well described in the manual</a>. The other two are both compiled into the same binary, the main function decides which component it should become after parsing the command line parameters. The common part</h2> There is a small common part that always happens when the greetd binary starts up: It invokes the commandline and configuration parser to get its complete configuration.</li> It invokes mlockall()</a></code> to prevent sensitive data like buffers containing passwords from being swapped to disk.</li> Then it decides what to become.</li> </ol> The greetd session worker</h2> The simpler part of greetd is the actual session worker, so I'll start with that … The relevant worker()</code> function in greetd/src/session/worker.rs</code> </a> The session worker is started by starting greetd with the --session-worker</code> or simply -w</code> flag and passing the number of a file descriptor that is supposed to be a Unix socket for exchanging commands and feedback. What it does</h3> The session worker itself is one strand of pretty straightforward code that feels a bit like a shellscript that asks for input and aborts with an error message on failure. It does a few things: Wait for an InitiateLogin</code> or Cancel</code> message.</li> Setup a conversation with PAM</abbr>.</li> Let a PAM adaptor implemented in a separate file</a> do the authentication conversation.</li> Do pam_acct_mgmt()</a></code> and pam_setcred()</a></code> and send a Success</code> message.</li> Wait for a Args</code> (sets the command for ) or Cancel</code> message and answer Success</code>.</li> Wait for a Start</code> or Cancel</code> message.</li> Fetch user information from PAM.</li> Set a session id using setsid()</a></code> to make the process the root of a new session.</li> If we have a specific virtual-terminal configured. Tell PAM that we want to run in that tty.</li> Set the XDG_VTNR</code> environment variable via PAM.</li> Open the terminal file, set the tty to text mode, clear it and switch to it if necessary.</li> Use the greetd terminal abstraction to do the plumbing for the pipes</a> and switch the controlling tty</a>.</li> </ol> </li> Set the pwd</abbr> for the new session, either to the users home or the filesystem root as fallback.</li> Set a whole bunch of environment variables for the session (including the GREETD_SOCK</code> for the greeter, and the ones received with the Args</code> message) using PAM.</li> Prepare the command to execute in the new session (shell + profile files + session command)</li> Fetch the environment variables for the new session from PAM.</li> Fork</li> </ol> After the fork the parent sends the pid</abbr> to the the greetd server process and closes the socket, sets its parent death signal</a> to SIGTERM</code> and waits for the child thread to terminate before ending the PAM session. The child process drops its privileges, also sets its parent death signal</a> and then uses execve()</a></code> to launch the prepared command with the environment provided by PAM. Note on Environment variables: The variables received using the Args</code> call get set before the ones coming from the greeter to avoid overriding some important variables. This behaviour was introduced</a> after making it possible for the greeter to set the environment variables (again)</a>. Personal opinion: If you have a good reason for changing those you shouldn't be doing that through the greeter anyway. Messages between Session Worker and Server</h2> For later reference here are the messages that are exchanged between session worker and server via given socket using a JSON based protocol. Parent to Session Messages</h3> InitiateLogin</code></h4> Message sent to start the login process. string service </dt> The PAM service id so that PAM knows which configuration to apply </dd> string class </dt> ends up in the XDG_SESSION_CLASS</code>, either 'user' or 'greeter' </dd> string user </dt> the username to log in as </dd> bool authenticate </dt> set to false</code> if we have an autologin session </dd> TerminalMode tty </dt> The description of the terminal to attach to </dd> bool source_profile </dt> Whether or not to read the profile files before executing the session command </dd> </dl> PamResponse</code></h4> A response to a question from PAM. string? response </dt> The response, may be null </dd> </dl> Args</code></h4> Sets the command to be executed for the session when Start</code> is sent. string[] cmd </dt> An array of arguments representing the command </dd> string[] env </dt> An array of environment variables to pass to the given command (it was added (back) on 2022-08-13</a>) </dd> </dl> Start</code></h4> Gives the final go after the session is full set up. Cancel</code></h4> Can be used at almost any step to cancel the session setup. Session to Parent Messages</h3> Success</code></h4> What it says on the tin. Signals that a command has successfully finished. Error</code></h4> Signals that an error has been caught somewhere. Error error </dt> A rusty error response, probably gets serialised to an error message </dd> </dl> PamMessage</code></h4> A Question or other Output from PAM. AuthMsgType style </dt> an enum telling what kind of message this is and what the expected response is, one of Visible, Secret, Info or Error </dd> string msg </dt> a string intended for the human trying to log in </dd> </dl> FinalChildPid</code></h4> Sends back the process id of the session child after the session was launched. uint pid </dt> the process id </dd> </dl> The greetd server</h2> The server process is the one started by the init-system, it is responsible for launching the session workers and telling them what to do, it also houses the internal state machine that knows which session to launch next, it also does some "managing" for the processes in the session. The server is made up of multiple components: A base to orchestrate the startup and running sequence, also handles signals</li> A Context</code> and and ContextInner</code> to keep track of multiple sessions and scheduling</li> Session</code> objects that launch the session workers and handle the communication</li> A function that does the communicating with the greeter</li> </ul> What it does …</h3> … on startup</h4> The entry point is the main()</code> function in the greetd/src/server.rs</code> file </a> After the initialisation has decided that it will be a greetd server the main function also does a few things: Determine the service name it will tell PAM based on which configurations for PAM are available. (warns when using the login</code> fallback)</li> Get some info about the user configured for the default_session</code></li> Create the Unix Socket (Listener</code>) for the greeter, set permissions and set the GREETD_SOCK</code> environment variable.</li> Get information about the tty it is supposed to be on. Returns an error told to wait for the terminal to become active it waits (terminal.switch</code> set to false</code> in config).</li> Create a Context</code> to do the housekeeping for us.</li> Start initial session or greeter for the default session depending on which one is configured. (Other state transitions after initial bootstrapping are handled by the Context</code>).</li> Create the runfile to know the initial run apart form the others in terms of system restarts, not greetd restarts.</li> Loop to handle posix signals (SIGALRM</code>, SIGCHLD</code>, SIGTERM</code> and SIGINT</code>) as well as Incoming connections on the greeter socket.</li> </ol> … on an incoming connection from the greeter</h4> The connections are accepted using the main-loop (see previous section) the handler gets the bidirectional datagram stream and a copy of the Context</code> structure (the inner context staying the same, this is how the information about the current global state is exchanged between possibly multiple connections). The client_handler()</code> in greetd/src/server.rs</code> handles the communication </a> Inside this function is a loop that reads the request and then decides which function to call on the Context</code>, wraps up the response and sends it back. CreateSession </dt> Calls the create_session()</code> function with the given username, if successful, responds with the first question </dd> PostAuthMessageResponse </dt> Calls the post_response()</code> function to answer the PAM question, fetches and returns the next question or the Ready</code> response if that didn't fail </dd> StartSession </dt> Calls the start()</code> function and returns the result </dd> CancelSession </dt> Calls cancel()</code> function and returns the result </dd> </dl> There are two helper functions: client_get_question(context) </dt> This function fetches the next question from the context using the get_question()</code> method, if it returns a question it packs that into an object that can be serialised and sent back, if not it calls wrap_result()</code> on the result and returns that, meaning automatic error handling or an automatic success message in case of an ok. </dd> wrap_result(result) </dt> This function takes a Result</a> and spits out a spendable success or error message containing appropriate information about the error. </dd> </dl> Context</h2> The Context</code> mechanism is implemented in greetd/src/context.rs</code> </a> As already stated greetd manages its state using Context</code> objects (no idea what the correct rust terminology is, but its a data-structure with a set of functions and quacks like an object). To be more precise it uses two context objects, the outer Context</code> for configuration and connection state and the ContextInner</code> for storing global state of a 3-stage session pipeline that is shared between all interested parts of greetd. Datastructures</h3> The Context</code> object holds the following values: RwLock</a><ContextInner> inner </dt> The ContextInner</code> object for global state wrapped in a lock that allows shared reading or exclusive writing </dd> string greeter_bin </dt> The command for the greeter </dd> string greeter_user </dt> The user the greeter runs as </dd> string greeter_service </dt> The PAM service name for starting greeter sessions </dd> string pam_service </dt> The PAM service name for starting normal sessions </dd> TerminalMode term_mode </dt> The vt in which this context spawns sessions </dd> bool source_profile </dt> The source_profile</code> setting </dd> string runfile </dt> The path of the runfile for persisting sate across possible greetd restarts </dd> </dl> The ContextInner</code> has the following fields: SessionChildSet? current </dt> Information about the currently running session </dd> SessionSet? scheduled </dt> The session that will be started once the currently running session terminates </dd> SessionSet? configuring </dt> Stage before scheduled</code>, a session that isn't ready to be launched just yet because the greeter is still answering the questions PAM has </dd> </dl> The SessionSet</code> here is a Session</code> (representing a session that is being set up) with a timestamp attached. The SessionChildSet</code> stores a SessionChild</code> (representing a running session) and has an is_greeter</code> flag in addition to the timestamp. Note: There is a distinction between the Session</code> and SessionChild</code> object because a session that is already running doesn't care about all of the communications foo and is only interested in finding out if the session is still running and ending it if it isn't supposed to be running anymore, the only shared value is the pid of the session worker. Methods</h3> The methods on the context seem to be made to make it easy to write down what should happen from the outside, also they are pretty well commented so you know how they are supposes to be hooked up. start_unauthenticated_session()</code></h4> Directly start an unauthenticated session, bypassing the normal scheduling. </blockquote> This one translates to spawning a session worker and sending it all commands that should be needed for an autologin session and gives empty answers for any questions PAM asks. This is used for the initial_session</code> and the sessions for the greeters. It returns the created Session</code> as an object. Username, session-class, PAM-service and the command are given as arguments. start_greeter()</code></h4> Directly start a greeter session, bypassing the normal scheduling. </blockquote> A small convenience wrapper that preconfigures the start_unauthenticated_session()</code> with stored settings for a greeter session, uses the greeter_service</code> as the PAM service. It also returns the created Session</code> object. greet()</code></h4> Directly start a greeter session, bypassing the normal scheduling. </blockquote> This one starts a greeter session and sets it as the current</code> session in the inner context. It returns an error if that current</code> session is already occupied. start_user_session()</code></h4> Directly start an initial session, bypassing the normal scheduling. </blockquote> This will use start_unauthenticated_session()</code> to autologin a given user with the given command. (used to implement the initial_session</code>) create_session()</code></h4> Create a new session for configuration. </blockquote> Creates a session preconfigured with the Context settings and the given username and swaps it into configuring</code>. Also mitigates a race-condition by cancelling a swapped out session. Returns an error if there is no current</code> session or there already is a session configuring</code> or scheduled</code>. cancel()</code></h4> Cancel the session being configured. </blockquote> get_question()</code></h4> Retrieve a question from the session under configuration. </blockquote> Uses the get_state()</code> method on the Session</code> and returns either an ok when there are no more questions (Ready</code>) or the question that is currently open. post_response()</code></h4> Answer a question to the session under configuration. </blockquote> Calls the post_response()</code> on the currently configuring</code> Session</code>. start()</code></h4> Schedule the session under configuration with the provided arguments. </blockquote> Returns an error if the configuring</code> session has not signalled that it is ready yet or isn't present. Sets the sessions command to the given value, and swaps it into the scheduled</code> stage, a session that already was in scheduled</code> will be cancelled. Sets a timer to fire the SIGALRM</code> signal in five seconds which will be received using the alarm()</code> method. alarm()</code></h4> Gets called by the main-loop when a SIGALRM</code> (timer) arrives. Is a noop if no session is scheduled. If the current session is still running it will be sent a SIGTERM</code> or SIGKILL</code>, depending on how much time has elapsed since the session was scheduled. The SIGALRM</code> will be set up to fire again in one second. If the current session is cleared the seduled</code> session will be sent a Start</code> command and be turned into the current</code> session. check_children()</code></h4> Gets called by the main-loop when a SIGCHLD</code> (a child process has terminated, usually) arrives. This function calls the waitpid()</a></code> function with the NoHang</code> flag set to tell it that we don't want to wait if there is nothing new it can tell us. It can react to a bunch of events: In case of a StillAlive</code> or an error indicating that we currently don't have any children the function exits. </li> States indicating some kind of normal event but not matching as well as Interruptions get thrown away. </li> Something exited or was killed with a signal and the current</code> session owns that pid: If a session is scheduled</code> we try to start it and make it the current</code> session, if not and the session was our greeter return an error, in case of a normal session use start_greeter()</code> directly and make it the current</code> session. </li> </ul> terminate()</code></h4> Notify the Context that we want to terminate. This should be called on SIGTERM. </blockquote> Shuts down the configuring</code>, scheduled</code> and current</code> session while holding the write lock to avoid race conditions and sessions being rescheduled. State Machine</h3> All these functions form the state machine that cycles between greeter and user sessions and makes sure the pipeline from configuring</code> to current</code> doesn't get clogged up by anything misbehaving and that greetd cleans up after itself. The Session Interface</h2> The session interface is implemented in greetd/src/session/interface.rs</code> </a> To make the external session processes and the context talk to each other Greetd has the Session</code> class. It wraps an entire session process and handles the communication. It also remembers the last incoming message as last_msg</code> and the session processes pid</code>. However, after setting up the session (as noted earlier) the Communication channel becomes useless as we now only care about the sessions current running status detecting when it exits and ending it if it misbehaves. This is the reason a second object, the SessionChild</code> exists which holds the session managers pid as task</code> and the pid of the process in the session as sub_task</code>. Methods of Session</code></h3> new_external()</code></h4> Creates the Session</code> object and spawns a session worker process with a datagram socket attached for communication. initiate()</code></h4> Send an InitiateLogin</code> message to the session worker. get_state()</code></h4> Tries to receive a message from the session worker if there is no last_msg</code> and stores it in last_msg</code> Then it returns a SessionState</code> struct contain either a question from PAM or a Ready</code> if PAM has no further questions. This is used in the Context.get_question()</code> method. cancel()</code></h4> Sends a Cancel</code> message and resets last_msg</code>. post_response()</code></h4> Send a response to an authentication question, or None to cancel the authentication attempt. </blockquote> Sends a PostAuthMessageResponse</code> message and resets last_msg</code>. send_args()</code></h4> Send the arguments that will be used to start the session. </blockquote> Since 2022-08-13</a> this also sends a list of environment variables to match the new Args</code> message. Also reads a message after that and returns whether the command was successful. start()</code></h4> Sends a Start</code> request and waits either for an Error</code> or a FinalChildPid</code> answer while dismissing questions from PAM with empty answers. After that it shouts down the socket and returns a SessionChild</code> object as the Session</code> object mainly made for session setup is no longer useful once the session is running. Methods of SessionChild</code></h3> owns_pid()</code></h4> Tests if the task</code> equals the given process id. (Used for checking if the session terminated on a SIGCHLD</code> signal in the Context</code>) term()</code></h4> Used to send the session process (sub_task</code>) a SIGTERM</code> to tell it that we want it shut down. kill()</code></h4> Sends a SIGKILL</code> to both, the session manager and the session process. Used by the Context.alert()</code> method when a session takes too long to shut down. Updates to this Page</h2> 2022-09-11</h3> Added information about the commits adding back the environment variables and another round of spellchecking. May also have broken some links (I hope not too many). 2023-03-03</h3> Added Link to Kenny Levinsens blogpost. But Why?</h2> The reason I created this writeup is to first of all have an understanding of what my login manager of choice does and also to have a reference when tinkering with it. One of my plans with this includes making greetd capable of running multiple sessions simultaneously to allow user switching and to be able to reuse the greeter for the initial login for locking my screen. In case you have found a mistake, have questions or just feedback in general … … please contact me! </a> Building a crappy cheap LightDM 1/? 2022-08-05T00:00:00+00:00 This is a lightly edited mind dump, don't expect a well written post here. What it is, does and why I built it</h2> The goal of this little project was to build a collection of tools that is able to switch between two sessions on a Linux system for the purpose of having a session for working and one for logging in and unlocking the screen, the way it is implemented is by changing virtual terminals</a> to the corresponding session when certain state-changes occur, much like LightDM</a> and the light-locker. The reason I'm doing this is because I want to be able to replace my currently pretty barebones screenlockers with a rich screenlocker that is able to do more than just showing me an unlock form, mostly for my phone but also for my other machines. Why not just use LightDM?</h3> Maybe oversimplified answer: I have the opinion that LightDM isn't great on mobile. Implementation</h2> The core of this contraption are two instances of greetd</a>, one for actually logging in and one for giving me a properly set up session to run my compositor in which could be replaced by an instance of tinydm on my phone. These two communicate by always running one of two scripts the locker and the unlocker, both grab the current vt with the chvt</code> command, write to pidfiles and kill</code> each other and run hooks on exit. Please Note: For my current implementation the user that runs the logged in session needs passwordless sudo access to the chvt</code> command, in a multiuser scenario this may not be the best idea. Unlocking</h3> Unlocking works by entering your credentials in the greeter, triggering the unlock</code> script which sends a SIGUSR1</code> to the locker process (using the pidfile), and then entering a sleep loop waiting for the next session lock. The lock</code> script handles this SIGUSR1</code> by killing the lockscreen (in my case swaylock) and using chvt</code> to switch the currently active terminal to the one of the real-work-session. Locking</h3> Locking works almost exactly the reverse way as unlocking. The lock</code> script starts the screenlocker, kills the unlocker, this time with an ordinary SIGKILL</code> and starts waiting for being unlocked. The unlocker</code> uses chvt</code> to grab the vt with the greeter on it and exits telling greetd to start the greeter session for the next unlock. Bootstrapping</h3> To solve our little problem of how to bootstrap this thing there is also an initial-session-lock</code>, a script that writes its pid to the lockers pidfile and only exits with a success code when it receives a SIGUSR1</code> to act as a trigger for starting the session on the first unlock. For shutting down my current solution is to wrap this in a loop and kill the unlocker after sway exits. Code</h2> Currently this is an experiment I've run on my personal dotfiles so the sources are not really in any central location yet and you probably don't want to just copypasta my scripts. Scripts</h3> My personal screen-lock</code> script</a></li> My unlocker</code> script</a></li> My initial-session-lock</code> script</a></li> </ul> Greetd configuration</h3> As mentioned above I have two instances of greetd running, one that starts the unlocker on login and one that starts the actual session. A pretty standard greetd configuration with the unlocker</code> script instead of a real session.</figcaption> [terminal] # The VT to run the greeter on. Can be "next", "current" or a number # designating the VT. vt = 7 # The default session, also known as the greeter. [default_session] # Starting tuigreet here which starts the unlocker command directly to unlock the second session command = "tuigreet --cmd unlocker -r -t" # The user to run the command as. The privileges this user must have depends # on the greeter. A graphical greeter may for example require the user to be # in the `video` group. user = "_greeter"</code></pre> </figure> The one starting the actual session could be replaced by something like tinydm</code> as I only care about the autostarting part and that the environment is properly set up so I don't have to do it myself here. A greetd configuration that uses the `initial_session` mechanism to autostart a shell oneliner that starts our session.</figcaption> [terminal] # The VT to run the greeter on. Can be "next", "current" or a number # designating the VT. vt = 8 [general] # Don't know if actually necessary but I don't want to restart the session I'm writing this post in runfile = "/run/slatian-greetd-initial-session" [initial_session] # The command that wits for the initial unlock, if successful starts the session and returns the control to the lockscreen after the session ends before it starts over. command = "sh -c 'while true; do initial-session-lock && dbus-run-session sway; kill \"$(head -n 1 \"$XDG_RUNTIME_DIR/unlocker.pid\")\"; done'" user = "slatian" # default session omitted … nothing interesting there</code></pre> </figure> Sudo configuration</h3> The following snippet placed in a file in /etc/sudoers.d/</code> will give everyone in the wheel</code> group (the group that has sudo access anyways, may also be called sudo</code> on some systems) the permission to run chvt</code> without having to enter a password. %wheel ALL= NOPASSWD: /usr/bin/chvt</code></pre>Almost EOF …</h2> More or less obviously this is very hacky, has multiple issues and falls apart very easily, somewhere in the next few months I'll try to find a cleaner way to implement this without relying on too much on shell scripting and firing signals at processes using pidfiles. Until that happens I'll continue to use my sxmo screenlocker</a> on my phone because it mostly works well enough for what I need. Tutorial: Smart ssh jumping 2022-06-05T00:00:00+00:00 I'm assuming you know what an ssh jump host is, why you want it and how you configure it in your ssh ssh_config</code> file. The problem I'm trying to solve</h2> SSH jumping (via JumpHost</code>) is usually useful when you have a bastion host you have to log into to reach the network behind. In some environments you never reach this "private" network in any other way, but if the private network belongs to your home- or office and the bastion is for connections from the big scary intertubes you want to use it only when necessary, meaning we need some kind of conditional ProxyJump</code>. Example scenario</h2> Lets assume we have the following two hosts: tesla </dt> This is our bastion host publicly reachable </dd> hawking </dt> This is our private host we actually want to connect to </dd> </dl> The easy workaround</h2> The easy solution would be to have multiple aliases and using your brain to make the decision by using the hostname when at home and the hostname-remote when elsewhere. Example ssh configuration with different aliases for local and remote connections:</figcaption> Host tesla HostName ssh.example.org User nicola IdentityFile ~/.ssh/id_foo Port 22222 Host hawking User stephen IdentityFile […] Host hawking-remote ProxyJump tesla User stephen IdentityFile […] HostName hawking Port 22</code></pre> </figure> Problem is that you have to type the remote alias every time you're not at home. Also using git this way is pretty annoying (you have to set up multiple remotes and manually tell git which one to use). Smart solution</h2> The smart solution would be that ssh somehow could figure out if it is "at home" and if not enable the proxy jump option. For that we have to answer the following two questions: Are we at home?</li> How do we tell ssh to (not) use the ProxyJump?</li> </ul> Are we at home?</h2> To find out if we are at home there are several options: Querying DNS and for some address that is different in our home network - Too complicated to set up and is a PITA</abbr> with slow resolvers (ssh connections already take 3 seconds to establish on school WiFi 🙄)</li> Comparing the WiFi name - This is not an option for wired networks.</li> Manually - forget it I wanted to get rid of that</li> </ul> All of the above are not really desirable, so lets think of something else … One thing that is usually frowned upon is fingerprinting, however if we turn the concept around that your local machine fingerprints its environment all of the sudden it becomes a pretty good tool. If the year is somewhere after 2020 you probably have IPv6 configured and if you are someone like me, you probably have your own local prefix which is a pretty unique identifier for a network which the router is nice enough to tell your device when it connects! So the magic for finding out if you are at home becomes an ip a</code> piped into a grep -F "$ip_address"</code>. For my purposes I wrote a little script called has-ip-grep</code> that is a simple shell pipe returning success if a given "fingerprint" matches and fails if it does not. The has-ip-grep</code> script which tests if the first argument given to it matches the IP-address output part of an ip a</code> command.</figcaption> #!/bin/sh set -e ip a | awk '/inet/ { print $2 }' | grep -F "$1"</code></pre> </figure> Explanation: ip a</code> prefixes the addresses assigned to interfaces with inet</code> or inet6</code> the awk command therefore looks for lines containing the string inet</code> and if it finds one outputs the second column which is the IP-address. The output gets piped into a silent grep command that is told to look for instances of the first script argument if that's not the case. The set -e</code> makes sure the script returns an error code if that match fails. Examples</h3> In the first example we try to match against our local IPv4 loopback interface which should always return a success, which works great for a quick test! has-ip-grep 127.0.0</code></pre> Example 1: Succeeds because we have the IPv4-loopback-address assigned to our lo</code> interface.</figcaption> </figure> The second example show matching against the prefix usually used by a popular series of home routers made by a German company, this is also great for testing but it will very likely not be enough to identify your home network because your neighbours and your friend's network probably have the same fingerprint. has-ip-grep 192.168.178</code></pre> Example 2: Usually succeeds if you are connected to a popular series of German home routers.</figcaption> </figure> Other problems are that because we match substrings here this will only be convenient to match against ranges that align nicely with the decimal notation and that we get false positives, for example if someone assigns us a 10.192.168.178</code>. We can reduce the probability of false positives by matching against our IPv6-address, replace the fd45:6789:abcd:</code> in the third Example with your own local IPv6-prefix. I don't recommend you rely your global unicast prefix as it is usually assigned by your ISP</abbr> and may change behind your back. has-ip-grep fd45:6789:abcd:</code></pre> Example 3: Will succeed in your home network and maybe some other random network half around the globe you will never know exists.</figcaption> </figure> So now we have a pretty good (works most of the time) way to know if we are at home lets put it to use! Conditional ProxyJump</h2> Luckily ssh has exactly what we need to combine it with our command we just created. You are probably familiar with the Host</code> keyword in your ssh configuration, turns out if you have a decently modern version of ssh there is also a Match</code> keyword which supports some arguments. See the manual: man 5 ssh_config</code> </a> With that knowledge you can match against the target host (if you don't you'll run into funny recursion …), the network fingerprint of your choice and arrive at a configuration that can decide on the proxy without having to query your brain. Example ssh configuration which automagically knows how to connect.</figcaption> Host tesla HostName ssh.example.org User nicola IdentityFile ~/.ssh/id_foo Port 22222 Host hawking User stephen IdentityFile […] Match Host hawking !Exec "has-ip-grep '192.168.42' && has-ip-grep 'fd42:f00b:aba2:'" ProxyJump tesla HostName hawking Port 22</code></pre> </figure> Explanation</h3> The configuration for tesla stays the same (assuming we only want to connect to it from the public side). The configuration for the local connection to hawking can stay too, the Host hawking-remote</code> directive was replaced with a Match</code> that tests if we want to connect to hawking and then executes our shell command. In this case we check for two IP-addresses that we expect to get assigned in our private network, As I'm running dual stack it won't hurt to check against the IPv4-address too. These two commands are combined with some shell syntax and quoted using double-quotes because that's how ssh_config</code> works. The Exec is prefixed with an exclamation mark because we want to match if we are NOT at home. The part after that is the same as with the remote alias except for the User and IdentityFile because we now the Host hawking</code> always matches and we don't have to duplicate them anymore. I can imagine it being a lot of fun configuring several Networks this way 🙃. Security</h2> This is by no means any kind of security measure, if the network you are connected to matches the fingerprint of your private network this will make the wrong decision to connect directly! However if you are running such a setup you should have the certificate fingerprints of your real servers already in your known_hosts file and then just be smart enough to not accept any new certificates over untrusted connections. If that finger print matches when you are not in your network (assuming you randomly chose an IPv6-prefix) you should get the hell out of there! Debugging</h2> ssh -v</code> is your friend! If ssh seems to do nothing except burning CPU cycles you might have ran into some recursion problem, Make sure that your Match</code> keywords always match on some host and that your proxy connections aren't jumping in circles! Also some pits I fell into (all of them are documented in the ssh_config</code> man-page): If your forget the Host</code> rule on a Match</code> you'll get some recursion going.</li> A Host</code> rule ends when a Match</code> rule starts.</li> When you assign a variable multiple times only the value from the first assignment is taken (especially important when you want to rewrite the host for short aliases and later to localhost when redirecting into a tunnel which won't work).</li> </ul> That's it!</h2> Have fun connecting to your servers without having to think about changing names! If you spotted a mistake somewhere or have other feedback feel free to contact me! Integrating swaylock-mobile with Sxmo 2/? 2022-05-07T00:00:00+00:00 Sxmo 1.9.0 was released last weekend</a> for PostmarketOS edge, which means … I have continued my work on the screenlocker as promised. In this part I'll mostly explain what needed to be done (and why) for the screenlocker to work. Related Resources</h2> Part 1 documenting how the screenlocking part of sxmo works under the hood</a></li> Screenlocker Integration Dotfiles</a></li> Swaylock mobile, my screenlocker</a></li> </ul> What my current solution can and can't do</h2> Current sate of swaylock-mobile is, that it's a pretty standard swaylock that fits on a touchscreen (even when rotated!) with a keypad for entering numeric pins without a real keyboard, that is enough to integrate it into Sxmo and get some additional security for my phone in daily use. So the integration currently handles: Locking the screen, before turning it off</li> Proximity-lock for both locked and unlocked states</li> State-management for the idle locker and lisgd</li> </ul> More important, what does it NOT handle: X integration, I have not tested it with the dwm version</li> Graceful degradation, fallback in case there is no screenlocker doesn't work</li> Maybe I have broken some suspend inhibitors … I did</li> Remaining slightly broken states</li> </ul> Changes</h2> New Wrapper script run-screenlocker</h3> This is a script that runs swaylock-mobile until it exits with a success status-code (the warning that you should make sure it is restarted in case of a crash still applies) and after that calls the unlock hook. This script gets called from the lock hook and should be the only thing that can call the unlock hook. (one exception would be the fallback …) Hooks that were modified</h3> sxmo_hook_inputhandler.sh </dt> The Powerbutton now only toggles screenon and screenoff and calls the lock hook before off, also the corner-swipe locks the screen and turns it off. </dd> sxmo_hook_lock.sh </dt> Just does, what it says on the tin: locking, also makes sure it only locks when called from an unlocked sate, also invokes the new idlelocker hook. </dd> sxmo_hook_proximitylock.sh </dt> The proximitylock now calls the screenon hook instead of the unlock hook, also disabled the can_suspend locks. </dd> sxmo_hook_screenoff.sh </dt> Now uses $SXMO_STATE.screen for storing screen state. </dd> sxmo_hook_unlock.sh </dt> Now uses the idlelocker hook for setting the screenlocker. </dd> </dl> Hooks that were added</h3> sxmo_hook_idlelocker.sh </dt> Sets the idlelocker for screenon scenarios, depending on the SXMO_STATE. </dd> sxmo_hook_screenon.sh </dt> Gets called for turning the screen on (and some screen related services). </dd> </dl> New state-file</h3> There is a new file to keep track of things, previously $SXMO_STATE was used for storing three states sxmo could be in: screenoff, locked and unlocked. These states would still be sufficient, but the model with a state for the lock (locked and unlocked) and a state for the screen (on and off), which allows me to simplify the previously error prone and complex state machine into two state-machines which just turn things on and off without having to make assumptions about the screen-state or the lock status. For that I needed a second state-file though which I called $SXMO_STATE.screen for now (clean solution would be to introduce a $SXMO_SCREEN_STATE and $SXMO_LOCK_STATE). New environment variable: SXMO_REASON</h3> Before rewriting the state-machines, I tried to work with the old states and figuring out what to do depending on where the call came from, which turned out to be a horrible buggy mess, but its the reason SXMO_REASON was introduced. Why is it still there you ask: Because if the screen is locked and the proximity-lock triggers in most of the cases you don't want the screen to turn on again by just uncovering the proximity sensor… the screenon hook script currently uses it to cancel if this variable is set to proximity-lock and the lockscreen is active. (Someone could make that a preference, however it won't work when the system goes to sleep) Additional sway configuration</h3> One thing that isn't immediately obvious is that you want to turn your screen back on again and that it doesn't work with the default configuration, the reason being that sway detects when a proper screenlocker is active it only allows keyboard bindings (the power-button behaves as if it was the XF86PowerOff key on a keyboard) when they are bound with a --locked switch, so I put a bindsym into my sway config file, the cleaner way would have been to update sxmo_swayinitconf.sh (I did it that way because I hope this way it will break less often than modifying the swayinitconf script). Fixing the remaining TODOs</h2> It doesn't work with X</h3> I (or someone who knows sxmo) would have to make sure all modifications are also applied to the dwm version (not sure about how to solve the power-button problem though) Graceful degradation in case of missing/disabled screenlocker</h3> My approach would be a dummy screenlocker, that creates a file somewhere and then watches it for close_writes with inotifywait (and deletes it after), the handler for the power-button in the file exists + locked + screen on would touch that file, which would trigger an unlock. Broken suspend inhibitors</h3> The proximity-lock hook by default locks the can_suspend mutex, I leave it on for the lock-screen which blocks the suspend script from doing its job, that's not battery friendly … Lets just bypass that by disabling the lock and hope it won't break. Slightly broken states</h3> There are two scenarios I know of where the state handling is slightly broken, both involve the proximity-lock: When unlocked + proximity-lock active (sensor covered) + power button the screen turns on for a second and then goes to lock + screenoff. I currently don't know why exactly this happens as the power-button should check against the screen-state and toggle it once, turning it on …</li> When locked + proximity-lock active + power button the screen turns on (like I want it when unlocked, ironically) and stays on for the 8 seconds timeout when not touched, this can be explained by how the proximity-lock currently triggers on changes (which is very good, because it saves power!)</li> </ul> Integrating swaylock-mobile with Sxmo 1/? 2022-03-18T00:00:00+00:00 I have been trying to introduce my touchscreen based screenlocker to Sxmo for over a week now. Turns out it is not as easy as I thought. In this part of the sersies I'll try to describe what sxmo does for screenlocking and what changes with the 1.9.0 release. The actual implementation will be discussed in part 2. What I have tried so far</h2> My first approach was to try out the desktop screensaver, the problem was, that it tried to interact with the newer version of Sxmo, so I reverted to messing with the default screenlock hook. This at first seemed simple, just insert the screenlocker after the service starting and stopping foo, lock my phone, the screenlocker comes up as expected, I try to enter my pin and … nothing, I forgot that Sxmo turns off the touch input for the locked mode, so I'm not able to enter my pin on the touchscreen. Ok easy fix turn the touch back on after starting the screenlocker … and yes I'm successfully able to unlock my phone, but for some reason the screenlocker comes up very soon and after coming Up my phone won't go to sleep … The reason turns out to be that the sxmo_screenlock.sh script figures out the current state by looking at what it turned off. That means, I broke that by messing with the touchscreen state, which brings us to the present. Where I'm now</h2> This is the "legacy" script that still runs on my phone which will be removed when Sxmo updates to 1.9.0. See the next section for the new solution. This script seems to be the pint where everything related to locking and sleeping goes through. Setting state</h3> It defines the following functions for setting state: lock </dt> Turns touch input off </dd> unlock </dt> Turns both touch input and the screen back on </dd> off </dt> Turns the screen off </dd> crust </dt> Sends the phone to deep sleep and determines why it woke up </dd> </dl> I probably want to to add some semantics here to find out why the locking/screenoff happened and launch my screenlocker if the reason is an idle session or a power button event. These functions get called from a few places: core/sxmo_inputhandler.sh </dt> Has a lock_screen_action() function for when the screen is locked to tun it off, into locked and back into unlocked. </dd> Also calls the lock function when you swipe away from the bottom left corner. </dd> core/sxmo_screenlock_deeper.sh </dt> Similar to the the lock_screen_action() above but is intended for being called after a few seconds of idle time and it won't move to an unlocked state. </dd> core/sxmo_proximitylock.sh </dt> This script periodically checks if the distance sensor is covered and then makes the decision to call the unlock or the off function, also based on the state of the screenlocker. </dd> core/sxmo_rtcwake.sh </dt> This script seems to be called by nothing … it calls the crust function on sxmo_screenlock.sh </dd> default_hooks/contextmenu </dt> Calls lock, off and crust with the intention of explicitly locking the screen. </dd> notifications/sxmo_notificationwrite.sh </dt> Calls the lock function if the screen is off, probably with the intention to make the screen light up when there is a notification. </dd> </dl> Idea from wiring that: start the screenlocker as a sxmo_service from a blocking-hook and make the unlock function refuse to work as long as it is running, execute the unlock when it returns, if no lockscreen hook is specified, fallback to turning off the touchscreen. Querying state</h3> These states from above can also be queried with the getCurState function, it returns: unlock </dt> When unlocked, this is returned if the touchscreen is on </dd> lock </dt> When locked with the screen on, this is returned if touch is off but the screen is on </dd> off </dt> When locked with screen off, this is returned when both touch and screen are off </dd> </dl> This will need some modifications as I'm breaking the assumption that touch state equals locked state. Quite a few scripts rely on this one: notifications/sxmo_notificationwrite.sh </dt> Tests for the off state to turn he screen on to get attention. </dd> core/sxmo_inputhandler.sh </dt> Uses this to cycle states. </dd> core/sxmo_screenlock_deeper.sh </dt> Uses this to determine the next deeper lock/sleep state. </dd> core/sxmo_proximitylock.sh </dt> (Ab)uses this to avoid unnecessary calls to unlock and off. But is not really context aware. It probably gets stopped elsewhere. </dd> core/sxmo_rtcwake.sh </dt> Cronjob, Only goes to crust if not in an unlocked state. (It has other factor too, but not being unlocked is a requirement) </dd> </dl> Looking at this I'm not really willing to break this function, maybe some kind of soft/hard lock mechanism for just turning off touch and having a proper lockscreen up would be useful. The new screenlock / suspend mechanism</h2> To understand the new mechanism you first have to understand sxmo_mutex.sh. (It is documented in it's first commit.</a>) Diff from previous, old mechanism: The is_idle and can_suspend hooks were replaced by mutex locks called from various sources.</li> The crust part of sxmo_screenlock.sh was replaced by sxmo_suspend.sh and sxmo_hook_suspend.sh.</li> The hacky detect locked state approach was moved to a much cleaner have a file which holds the state approach.</li> The rest of sxmo_screenlock.sh was replaces by hook scripts which makes my journey much easier.</li> </ul> Suspending</h3> This suspend part consists of a few files: default_hooks/sxmo_hook_suspend.sh </dt> Does the actual suspending. </dd> default_hooks/sxmo_hook_mnc.sh </dt> Figures out how long it is until the next Chronjob with sxmo_rtcwake in its name is scheduled. </dd> default_hooks/sxmo_hook_presuspend.sh </dt> Called before suspending, does cleanup like pausing media and turning off screen. </dd> default_hooks/sxmo_hook_postwake.sh </dt> Called after waking up with the reason for the wakeup as the first argument. </dd> core/sxmo_suspend.sh </dt> When called fires off the whole suspend hook firework, goes too sleep, wakes up again and does a bit of housekeeping. </dd> ore/sxmo_rtcwake.sh </dt> Cronjob wrapper (I assume) that does some housekeeping with locks to make sure the cronjobs don't get interrupted. </dd> </dl> The whole thing is triggered from the screenoff hook which starts a sxmo_periodically which runs sxmo_hook_check_state_mutexes.sh and if there is no good reason to stay awake goes to sleep using sxmo_suspend.sh with a sxmo_mutex.sh and a holdexec. (At least for touchscreens with three buttons which is the category my pinephone falls into) Locking</h3> Sxmo still has the three states, but they were moved to their own hooks. This means that … The unlock script is now responsible for calling sxmo_hook_lock.sh</li> The lock script is responsible for turning the screen off</li> The screenoff script is responsible for going to deep sleep</li> Unlocking still happens through the lockstate cycling in the inputhandler</li> The unlock script is called by the start hook</li> </ul> The proximitylock script calls the screenoff hook to lock and the unlock hook, so a screenlocker can actually be placed in the lock hook without having to worry about the proximitylock interfering, the postwake hook would have to unlock the touchscreen from the screenoff hook, which it already does and the lock hook shouldn't turn off the touchscreen (that's a beautiful hook fallback), the only "problem" with this approach would be that the state would say unlocked while the lockscreen is up. (interactive would be a better identifier) To be continued</h2> Note 2022-05-07: there was a previous version of this article that said: this little diff will do it, just use the lock hook … Nope, that won't work … Most of the above is still true for the released version (good enough as a little guide for myself at least). For the actual implementation see part 2 </a>

slatecave.net - Slatians Blog

The 39c3 Search Engines Post

Modern Web Search: Not Just an Index

What this post is about</h2> This post is abut the claim of:</p>

Being a Polite Crawler on the Web

Identify Your Crawler</h2> Identify your crawler so that the Admin on the other side knows who is crawling what and why. Crude attempts at trying to look like a Browser probably won't last long. This is mainly because your crawler has a very different goal from the average website visitor.</p>

Don't Scrape what You could get Through an API</h2> If you only need specific information that is available using a well documented API, strongly consider querying that API instead of scraping web pages.</p>

Don't Crawl Things you shouldn't</h2> While crawling the things you shouldn't crawl seems interesting and appealing it really isn't. In fact you probably want to crawl even less than you are allowed to.</p> Possible reasons for a Server to advise crawlers not to crawl certain sections are:</p>

A Forgejo-Runner with Podman on Alpine

Installing the Needed Dependencies</h2> Community repository needed:</b> Some of these packages come from the alpine community repository. Uncomment the relevant line in /etc/apk/repositories</code> and run apk update</code> to enable it.</p>

Creating a User for the Runner</h2> For the runner we'll create a regular user account without any special access. For this example I'm calling the user runner-for-someone</code>, you can of course use your own name, make sure you always replace runner-for-someone</code> with that.</p>

Configuring the Forgejo Runner</h2> For this step witch into the runner user using the following command:</p> sudo</span> -</span>iu</span> runner-for-someone</span></span></code></pre>Getting a Registration Token</h3> Now is a good time to obtain a runner token …</p> … for yourself:</p>

The Runner Service</h2> Like with the Podman service this will be split into a template and an actual service.</p> Again, resulting in one service per user.</p>

The Beginning …</h2> Now you have your own runner (hopefully 😅), it's up to you what you want to with it.</p> </blockquote>

Fixing synapse 1.128.0 with SQLite

You should read: Outrage Warps Reality

Today I learned: OpenLDAP

Who is this for?</h2> This is for people who know how to configure their LDAP clients, but now want some knowledge on how to configure a server.</p>

Understanding Lua Expressions - Live Posts

Thunderbird: Use S/MIME Certificates from LDAP

xdg-settings: Setting a default browser isn't that simple

Generic Browser configuration</h2> The generic browser configuration makes use of the already mentioned helper functions.</p>

Generic URL scheme configuration</h2> Generic URL scheme configuration is mostly wrapping around the already described helpers with some special treatment for the http</code> and https</code> schemes when the BROWSER</code> environment variables are set.</p>

Why not use && as a shortcut in shell scripting

What am I Talking About</h2> If you've been shell scripting for some time you have most probably seen the following construct:</p>

Fixing PWM flicker on the Arduino UNO R3

What happened?</h2> Since I'm trying to tell a story here, let's start at the beginning.</p>

Group based Permissions using polkit

Pretty Atom-Feed Previews with XSLT

xdg-mime: Mapping Files to Applications taken apart

Getting the MimeType of a File</h2> The info</code> action triggers a desktop detection and then decides which mime-type detection method to use. On success the output must be the mime-type of the file followed by a newline without any extra information.</p>

Installing and Uninstalling MimeType Descriptions</h2> Now that everything from getting the mime-type to the application is covered one may ask:</p>

"It just opens files" - xdg-open under the hood

Don't Scrape what You could get Through an API</h2>
If you only need specific information that is available using a well documented API, strongly consider querying that API instead of scraping web pages.</p>

The Runner Service</h2>
Like with the Podman service this will be split into a template and an actual service.</p>
Again, resulting in one service per user.</p>

Generic Browser configuration</h2>
The generic browser configuration makes use of the already mentioned helper functions.</p>

Generic URL scheme configuration</h2>
Generic URL scheme configuration is mostly wrapping around the already described helpers with some special treatment for the `http</code> and https</code> schemes when the BROWSER</code> environment variables are set.</p>`