Search This Blog

Showing posts with label shell. Show all posts
Showing posts with label shell. Show all posts

Tuesday, July 28, 2026

KeePassXC sync, Dropbox and RPi

Intro

I keep my passwords in the KeePassXC DB and use Dropbox to synchronising it between multiple computers (and phones). It has been working fine for years, until I decided to try set RPi as a desktop. Dropbox does not provide a client for Arm architecture. However, as quite often in the world of open source, there is an alternative. Rclone is a tool allowing you to manage files in over 70 cloud provider, including Dropbox.

Software is easy to install and configure. Just run rclone config and follow prompts, choose Dropbox, authenticate via browser, as documented in the Dropbox section of the project website.

Sync

For syncing one file, rarely changed, a simple script around rclone is good enough. It provides 2 functions: pull and push.

#!/bin/bash
REMOTE="dropbox:path/to"
REMOTE_PATH="${REMOTE}/YourDB.kdbx"
LOCAL_PATH="${HOME}/keepass"
LOCAL="${LOCAL_PATH}/YourDB.kdbx"

case "$1" in
  pull)
    rclone copy "${REMOTE}" "${LOCAL_PATH}"
    echo "Pulled from Dropbox"
    ;;
  push)
    rclone copy "${LOCAL}" "${REMOTE_PATH}"
    echo "Pushed to Dropbox"
    ;;
  *)
    echo "Usage: $0 pull|push"
    ;;
esac

The script assumes that the name of the remote was set to dropbox and the local copy is stored in the ~/keepass directory. Note, that for the copy source is the name of the file, but destation is the folder.

Cronjobs

To to mimize a chance of reading outdated passwords it's worth to automated the pull step. Just pull, because KeePassXC handles conflicts well at the application level (it can merge databases), but rclone itself is not conflict-aware. Automated push can overwrite changes from another machines, or copy a corrupted file.

# Automated sync every half-hour
*/30 * * * * rclone copy dropbox:path/to/YourDB.kdbx ~/keepass/ --log-file=~/.local/var/log/rclone-keepass.log --log-level ERROR
# Extra daily backup
3 3 * * * rclone copy dropbox:path/to/YourDB.kdbx ~/keepass/$(date +\%Y-\%m-\%d)/ --log-file=~/.local/var/log/rclone-keepass-backup.log --log-level ERROR

Links

Friday, May 01, 2026

Watching GitHub PRs

In one of my previous posts I explained how to check the status of your PRs from the command line. Recently, I added one more improvement to ensure that also PRs without any reviews yet are listed with the PENDING state. This is achieved by using the fallback operator // in the jq\(.reviews[-1].state // "PENDING") command.


gh pr list \
 --author @me \
 --json "number,title,reviews" \
| jq '.[] | "PR \(.number): \(.title) is \(.reviews[-1].state // "PENDING")"'

I also discovered (with some help from AI) how to watch the PR state live in full colour: combine the watch tool  --color option with one of 2 environment variables:


watch --color 'GH_FORCE_TTY=1 gh pr view'
watch --color 'CLICOLOR_FORCE=1 gh pr view'
  • GH_FORCE_TTY=1: is the gh specific option and forces the GitHub CLI client to behave as it is running in a terminal, even if it's not (e.g. piped or run inside watch).
  • CLICOLOR_FORCE=1: is part of the CLICOLOR standard and affects other tools as well. It forces the tool to output colours regardless whether it is running in a terminal or not. 

Links

Monday, March 30, 2026

More jq from journalctl

I've found another use case for `jq` when parsing service logs stored with journald. This time, I want to extract all non-INFO level logs from the service called slinky. In the previous example, I used awk to print only the part of a line that is valid JSON. However, sed might be better suited for this task. The following rule removes (replaces with an empty string) the beginning of the line up to the colon followed by a space ": ", which separates the timestamp from the log entry (JSON):

's/^.\+\]:\ //'

Examples

There are two examples of the command pipelines below. 

The first one checks the logs from the last 2 hours:


journalctl --since "2 hours ago" -u slinky.service\
 | sed -e 's/^.\+\]:\ //'\
 | jq 'select(.level != "info") '

The second one continuously prints new entries:


journalctl -f -u slinky.service\
 | stdbuf -oL sed -e 's/^.\+\]:\ //'\
 | jq 'select(.level != "info") '

The journalctl command is nearly identical in both examples (--since vs. -f). The jq select statement and the sed string replacement are the same. The main difference is that the latter uses the stdbuf command. It allows running the following command with modified buffering. The -oL option means that the standard output of the sed command is flushed line by line, enabling each entry to be passed immediately to jq.

Monday, March 02, 2026

Download GitHub Actions logs

I've been using GitHub CLI more and more lately. Recently, I had to debug a failing of GitHub Action run. Browsing long logs in the WebUI is a bit clunky, so I started downloading the full logs via the CLI. 

It is straightforward `gh` command if you already know the run number --  and that information can be obtained from the `gh` command with different options. 

I end up with following two-part shell snippet: 

VIEW=$(\
 gh run list \
 | grep $(git branch --show-current) \
 | head -1\
 | awk '{print $(NF-2)}') \
&& \
gh run view ${VIEW} --log > ~/Downloads/${VIEW}.log
 

The first command assign the run number to the VIEW variable. It parses the output of the `gh run list` command by:

  • filtering run for the current git branch (`grep`)
  • taking the most recent one (`head`)
  • extracting the run number (third column from the end via `awk`).

The VIEW variable is then used to fetch the logs for the specific run and save them to the uniquely named file in the Downloads folder.

Assumptions:

  • the workflow run belongs to the current git branch
  • it's the latest run for the branch 
  • The Downloads folder doesn't already contain a file with the same number (it would be overwritten)

Tuesday, January 06, 2026

Hash of the latest git commit

From time to time I have, not only know, but to paste the hash of the latest git commit somewhere else. I prepared a one-liner to get the hash in a format good to copy & paste:
git log --pretty=oneline |\
 awk '{print $1}'

Wednesday, November 26, 2025

Check status of your GitHub PR from CLI

This github CLI command (`gh`) will list each PR authored by you. The output contains the PR number, title and all reviews. They are printed as json, and piped to `jq`. From the output, jq filters out the id, title and state of each review and interpolate it into a string.

gh pr list \
 --author @me \
 --json "number,title,reviews"\
 | jq \
 '.[] | "PR \(.number): \(.title) is \(.reviews[].state)"'

Links

Monday, November 03, 2025

IP address oneliner

Command

The command to get a public IP from UNIX `ip addr` command:

ip -4 -o addr| \
 awk -F "( |/)" '$0 !~ " (10|127|172|192)\\." {print $7}'

The `ip` command options makes it to print only IPv4 addresses (`-4`) and output them in one line each (`-o`).

The `awk` app uses space and `/` as Field separators (`-F "( |/)`) and looks for lines without a substring containing one of 10, 127, 172, 192 numbers followed by a dot and preceded by space (`$0 !~ " (10|127|172|192)\\."`. It prints a seventh filed (`{print $7}`).

Example of ip command output

To better understand the pattern used in the `awk` below is an example of the output of the `ip` command.

1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
2: enp2s0f0np0    inet 56.112.121.59/32 metric 100 scope global dynamic enp2s0f0np0\       valid_lft 64241sec preferred_lft 64241sec
4: docker0    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0\       valid_lft forever preferred_lft forever

Wednesday, September 24, 2025

Json in logs and how AWK can help

Checking systemd service logs in a JSON format might be a bit annoying. The output is far from readable, but AWK comes to help. The key is to use the "{" character as a separator, and reusing it in the printf command.

In my case, I had to check the mev-boost outputting in JSON format and used the following command to make the output more readable. 

 journalctl -u mev-boost --since "2025-09-22 00:00:00" | \
  awk -F{ \
  '$0 !~ /io.ReadCloser/ { printf ( "%s%s\n" ,FS ,$2) }' | \
  jq .
 

The first line limits the output of journal logs to unit `mev_boost` from 22nd September 2025.  Second and third line direct AWK to split each line on the "{" character, and processes only lines which do not contain the string "io.ReadCloser". The lines with the string are not proper JSON. Next, the second field/substring is printed, but prefixed with the field separate ("FS"). Otherwise, the output will not be a valid JSON, because the opening "{" is used a separator. Such formatted line is then sent to jq. In the example, jq just prints the output in a human friendly way, but it can be used to query the output.

Thursday, August 28, 2025

Check the time of the current block in a Cosmos blockchain

echo BLOCK: $(curl -s 127.0.0.1:26657/status |\
 jq -r '.result.sync_info.latest_block_time');\
echo "NOW:   $(LC_TIME=C date -u +%Y-%m-%dT%X.%NZ)" 

Blockchain perspective 

If you deal with a Cosmos based blockchain, this command (actually there are 2 commands) helps to visualise the difference between the time of the latest block and current time. That's helpful if a node needs to catch up (e.g. after restarting from a snapshot). 

UNIX perspective 

The command has a few interesting "UNIX features". First, the curl from the localhost is silent (-s option) to avoid showing download stats. Its output is redirected to jq, which presents the raw (-r option) the time of latest block (.result.sync_info.lastest_block_time field). 

The second command returns the current time. The LC_TIME=Csets the locale to a standard, non-localized format, ensuring the output is always in a 24-hour clock and not a 12-hour AM/PM format, which can vary by location.  The option -u forces the output in the UTC, rather than the local time. Finally, +%Y-%m-%dT%X.%NZ formats date identical like the cosmos endpoint. 

Tuesday, July 15, 2025

Take me to the git home (and checkout me back)

I'm moving quite a lot around git projects. For some time, I have been wondering if there is a way to have an equivalent of argumentless `cd`. But rather of taking you back $HOME, changing directory to the git top-level one. I decided to make a deeper dive into the problem and ended up with the following function in zsh:

  function cdt ()
   cd $(git rev-parse --show-toplevel)}

Originally, I had an alias, but it did not work as designed. It took a root of git repository when sources (usually, during shell start). For a refernce that is my original alias.

  alias cdt="cd $(git rev-parse --show-toplevel)"

Another "dimension" of git project you can move around are branches. Last year, I learnt that the `-` works in the `git checkout` the same as in the `cd` command. So if we switch between branches, try:

  git checkout -

Sunday, February 04, 2024

Open file from command line (in Linux and Macos)

One of the nice feature of MacOS is the open command. It allows opening files directly from the command line without knowing the application linked to the file type. For example:

 open interesting.pdf 

opens the interesting.pdf file using whatever program is assigned to open PDF files. (If you want more example about the open command, you can check this link.)

For some time I wonder about a Linux equivalent. Recently decided to look for it more actively and check if AI might help. It did, and pointed at the gio command from the Gnome Input/Output library. After adding a function or an alias following block of code to the .zshrc, I have a Linux equivalent.

  •  alias

open="gio open"

  • function:

open () {

    gio open $1

}   

And finally, xdg-open is an alternative for the "gio open".

Saturday, December 30, 2023

Glow the Grip of MD files (from GitHub)

If you will every need to locally render an MD file, e.g. reading some documentation, you can use the glow [1] program. It renders a MD file in the terminal.

In the case of GitHub repository, an alternative is to use the grid [2] project. It sets a local webserver using the GitHub markdown API. It produces a local view of MD files as they would be in the GitHub website.

 

Links

  1. https://github.com/charmbracelet/glow
  2. https://github.com/joeyespo/grip

Sunday, November 12, 2023

Summary of a Terraform plan output

One of the most annoying thing when working with terraform is the size of output of the terraform plan command. For more complex environments, it easily can get to many thousand lines, even for what seems to be a small change.  It makes very hard to confirm that a code change does not have a side effects.

It would be nice to have the summary option, showing only resources and modules changed. I guess one day such feature will be added. In the meantime, I thought to use the grep command on the terraform plan output. It wasn't easy, because the output contain a few control character. After quite a few attempts, I found that following regex is a substitute.

terraform plan | grep -E "^[[:cntrl:]][[:print:]]+[[:space:]]+#\ "

Wednesday, May 17, 2023

How to find s3 bucket in multiple accounts (with awk and multiple field separator)

Imagine you have quite a few AWS accounts. In one of them, you don't know which one, there is an S3 bucket. The AWS CLI with awk and zsh can help to find it.

In the first step, let's prepare a list of all accounts, or rather profiles from the AWS CLI config (the ~/.aws/config file).

accounts=($(awk -F "( |])" '/profile sso/ {print $2}'  ~/.aws/config))

In the example, we limit the list only to profiles with the prefix "sso". The command uses awk to find any line with the string "profile sso" and print the second field from it. However, it does no use the standard field separator. There are 3 characters working as a separator: space, "|" and "]". Please also note the awk command is two pairs of "()".

 The list is saved into the accounts variable and used in the second command. It lists all s3 buckets from each account, and grep for the selected string, which of course can be the whole bucket name.

p=sso-prod 
bucket=my-company-not-so-important-bucket
for accounts ($accounts) {echo $account; aws s3 ls --profile $p| grep $bucket}

Thursday, November 04, 2021

dmidecode - the command I always forget about

 

From time to time I need to check some details of the hardware in one of my Linux server. There is a good command to do this, which I know exists and have good functionality, but cannot remember the actual text to call it. The command in question is:

dmidecode

And below links with examples and explanation how to use it:

  •  https://www.ubuntupit.com/simple-and-useful-dmidecode-commands-for-linux/
  •  https://linuxiac.com/dmidecode-get-system-hardware-information-on-linux/

Saturday, May 01, 2021

ABCDE in Crux

After my OS update to Crux 3.6 (3.6.1 to be precise) I cleaned non-main (core, opt, xorg) packages. One of the side effects is that I lost abcde. It was removed from the contrib collection, because of inactive maintainer. Along abcde, the cd-discid was also deleted from the same reason. I decided to add them to my port collections (https://wawrzek.name/crux/repo/). I started from the old contrib ports. Looking at sources I noticed that there is a recent patch for cd-discid. I included it into my port. I also encounter problems in running abcde with my config. There were missing musicbrainz Perl modules, so I added ports for them as well.

Thursday, October 22, 2020

Kernel module info

A few commands to help in checking and adjusting kernel module options: 

  1. Display information about kernel module, including all options

    modinfo $KERNEL_MODULE

  2. Display information about current value for each option

    for i (/sys/module/$KERNEL_MODULE/parameters/*) {\
      echo $(basename $i);\
      cat $i\
    }


  3. Change kernel module option without restart (temporary)

    echo "$NEW_VALUE" > /sys/module/$KERNEL_MODULE/parameters/$OPTION

  4. Load module with an option set to value
    insmod $PATH_TO_MODULE/$KERNEL_MODULE.ko\ $OPTION1=$VALUE1 \
    $OPTION2=$VALUE2


  5. Ensure kernel module option is set during boot up with grub
    1. Open /etc/default/grub
    2. Add/edit following line
      GRUB_CMDLINE_LINUX='$KERNEL_MODULE.$OPTION=$VALUE'

Of course $KERNEL_MODULE, as well as all other strings with "$" at the beginning, has to be replaced by appropriate value.

In CRUX /etc/default/grub does not exist, so it has to be created.

 Useful links

Sunday, October 11, 2020

Shell script to setup system with new kernel

 

This is a small script helping in setting up a new kernel. It copies files from the Linux kernel source directory and builds the matching initramfs file.

 

#!/bin/sh
#
set -x
if [ $(dirname $PWD) != '/usr/src' ]
then
	echo ""
	echo "Please change directory to a one with Linux kernel sources"
	exit 1
fi

version=$(basename $PWD |awk -F\- '{print $2}')


ls boot/vmlinuz-${version}-* 2> /dev/null && \
	build=$(ls -1 /boot/vmlinuz-${version}-*| sort -n | awk -F\- '{print $3}') || \
	build=1

release=${version}-${build}
cp System.map /boot/System.map-${release}
cp arch/x86/boot/bzImage /boot/vmlinuz-${release}
mkinitrd /boot/initramfs-${release}.img ${version}

grub-mkconfig > /boot/grub/grub.cfg

Wednesday, August 26, 2020

How to find a word in files spread over many subdirectories

Imagine that you need to try to find a word in hundreds files located in a plenty of subfolders. grep -Rc might be a good candidate, but it's going to print the list of all files and the word is in just a handful of them. To make the output better grep can be joined with awk (and in the example below with zsh). 

I needed to find an example of sqlcmd command usage in my Ansible roles collection. I used the power of ZSH globbing (a few other examples can be found here and in other places) to limit search to yml file only. ZSH "**" pattern is also a nice replacement to the grep "--recursive"option. In awk the "-F" flag replaces the standard delimiter with the ":" character and prints only lines with second field not equal to zero.

grep -c sqlcmd **/*.yml | awk -F":" ' ($2 != 0 ){print $0}'

Wednesday, April 29, 2020

Get Public IP address of Azure VM from shell on VM - part II

Last year I wrote a quick note with a method of getting Azure VM public address from the shell on VM. Recently I spent a bit more time to look at https://docs.microsoft.com/en-us/azure/virtual-machines/linux/instance-metadata-service and found more precise call to get IP address from the metadata url: 

curl -H Metadata:true \
 "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-08-01&format=text"

BTW. The same method allows to get a private IP address.

curl -H Metadata:true \
 "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2017-08-01&format=text"