Search This Blog

Saturday, December 30, 2023

Glow the Grip of MD files (from GitHub)

If you will every need to locally render an MD file, e.g. reading some documentation, you can use the glow [1] program. It renders a MD file in the terminal.

In the case of GitHub repository, an alternative is to use the grid [2] project. It sets a local webserver using the GitHub markdown API. It produces a local view of MD files as they would be in the GitHub website.

 

Links

  1. https://github.com/charmbracelet/glow
  2. https://github.com/joeyespo/grip

Sunday, November 12, 2023

Summary of a Terraform plan output

One of the most annoying thing when working with terraform is the size of output of the terraform plan command. For more complex environments, it easily can get to many thousand lines, even for what seems to be a small change.  It makes very hard to confirm that a code change does not have a side effects.

It would be nice to have the summary option, showing only resources and modules changed. I guess one day such feature will be added. In the meantime, I thought to use the grep command on the terraform plan output. It wasn't easy, because the output contain a few control character. After quite a few attempts, I found that following regex is a substitute.

terraform plan | grep -E "^[[:cntrl:]][[:print:]]+[[:space:]]+#\ "

Wednesday, May 17, 2023

How to find s3 bucket in multiple accounts (with awk and multiple field separator)

Imagine you have quite a few AWS accounts. In one of them, you don't know which one, there is an S3 bucket. The AWS CLI with awk and zsh can help to find it.

In the first step, let's prepare a list of all accounts, or rather profiles from the AWS CLI config (the ~/.aws/config file).

accounts=($(awk -F "( |])" '/profile sso/ {print $2}'  ~/.aws/config))

In the example, we limit the list only to profiles with the prefix "sso". The command uses awk to find any line with the string "profile sso" and print the second field from it. However, it does no use the standard field separator. There are 3 characters working as a separator: space, "|" and "]". Please also note the awk command is two pairs of "()".

 The list is saved into the accounts variable and used in the second command. It lists all s3 buckets from each account, and grep for the selected string, which of course can be the whole bucket name.

p=sso-prod 
bucket=my-company-not-so-important-bucket
for accounts ($accounts) {echo $account; aws s3 ls --profile $p| grep $bucket}

Tuesday, August 30, 2022

Network debugging in k8s

 

 Sometimes it is good to look at a network from the inside of a kubernetes cluster. Well-prepared images do not have any useful networking tools (because they are minimal image designed to do specific task). Therefore, you might need to run commands from a dedicated container. An example of such tool is the netshoot image. It can be found in GitHub:  https://github.com/nicolaka/netshoot

It's very easy to run: 
 

  kubectl run tmp-shell --rm -i --tty --image nicolaka/netshoot


Tuesday, May 31, 2022

Jenkins and Splunk (Cloud)

There is the Jenkins Plugin to set log export to Splunk. In Splunk Marketplace you can find the Splunk App for Jenkins, but the app works only for Splunk Enterprise, not the Cloud edition. Documentation describe how to connect Jenkins to Splunk. It includes values for cloud deployment as well as on-premises one. Thanks to that, the connection was easy to establish, but there were no logs from Jenkins. I had to manually created 4 indexes (I got them from this discussion) to get Jenkins logs visible in Splunk.

Monday, December 27, 2021

Sims 4 (and Origin client) on Linux with Steam

Recently, I've spent some time trying to make Sims 4 running on Linux. (That's the way you might spend time when you have a growing up daughter.) The ProtonDB entry was (and still is) GOLD, so it was encouraging. However, it didn't work on the first computer I tried. Steam client showed the game ran, but there was no window. Neither for the Sims 4 game, nor for the Origin client. It was:

  • Distro: Mint 20.2
  • CPU: Intel i5-7400
  • GPU: Nvidia (Zotec GT 610 1GB)
  • Kernel: 5.4.0-88 (Ubuntu/Mint)
  • Drivers: Nvidia closed source 390.144
  • Mesa: 21.0.3
  • Proton: various versions
  • Sims 4 works: No

Then I tried on my Crux machine, and it worked fine. I don't remember if I had to add PROTON_USE_WINED3D=1. The machine had:

  • Distro: CRUX 3.6.1
  • CPU: AMD Ryzen 5 2400G
  • GPU: AMD RX 6600 XT
  • Kernel: 5.13.2
  • Drivers: amdgpu
  • Mesa: 21.2.2
  • Proton: 6.3.7
  • Sims 4 works: Yes

So I thought that maybe GPU from the original  machine was too old, or NVidia drivers were causing problems. I replace the Nvidia GPU an AMD FirePro V5900 card. It didn't help.

The next step was to run a test on my old Dell XPS-13 laptop. I installed the OpenSuse Leap 15.3 on it. I played a bit with Proton version, but in the end Sims started with the Proton 7.0rc2-GE-1, the latest release from the Glorious Eggroll branch.

  • Distro: OpenSuse Leap 15.3
  • CPU: Intel i7-4510
  • GPU:Intel i915
  • Kernel: 5.13.18-59.10-default
  • Drivers: intel
  • Mesa: 20.2.4
  • Proton: 7.0rc2-GE-1
  • Sims 4 works: Yes

In the same time, I realized that I could swap CPUs between machines. The Ryzen had the build in Vega GPU core, which was not in use. The first phase was to check that Sims 4 could start on CRUX without discrete GPU. It worked fine. Then I swapped the CPUs (with motherboards). The CRUX (with Intel CPU and AMD GPU) worked fine, but the Mint with AMD Ryzen (and integrated AMD GPU) still struggled. I tried using the Proton from GE branch, which helped me on OpenSuse. No luck.

  • Distro: Mint 20.2
  • CPU: AMD Ryzen 5 2400G
  • GPU: AMD Radeon Vega 11
  • Kernel: 5.15.6-1-default
  • Drivers: amdgpu
  • Mesa:21.3.1
  • Proton: 6.3.8
  • Sims 4 works: No

I started to consider that something wrong is with Mint. The Sims 4 problem seems to be the problem with Origin client.  The first relatively easy change was to use the KDE instead of Cinnamon as a Window Manager for the Mint system. It didn't help. Next was to install OpenSuse along Mint. The only change compared to the Dell laptop was to use the Tumbleweed rather than the Leap edition.

  • Distro: OpenSuse Tumbleweed
  • CPU: AMD Ryzen 5 2400G
  • GPU: AMD Radeon Vega 11
  • Kernel: 5.15.6-1-default
  • Drivers: amdgpu
  • Mesa:21.3.1
  • Proton: 6.3.8
  • Sims 4 works: Yes

This time it worked fine. No issues. In conclusion, there is something wrong with Mint, but Sims 4 and Origin client works fine on Linux with Steam.

Thursday, November 04, 2021

dmidecode - the command I always forget about

 

From time to time I need to check some details of the hardware in one of my Linux server. There is a good command to do this, which I know exists and have good functionality, but cannot remember the actual text to call it. The command in question is:

dmidecode

And below links with examples and explanation how to use it:

  •  https://www.ubuntupit.com/simple-and-useful-dmidecode-commands-for-linux/
  •  https://linuxiac.com/dmidecode-get-system-hardware-information-on-linux/

Tuesday, July 20, 2021

How to check Jenkins credentails

If you every need to check what are the actual password in Jenkins credentials check this small groovy script  by Tim Jacomb. I helped me to confirm that credentials were corrupted during saving and restoring with Configuration as a Code plugin.

To use it (and any other groovy script). In your Jenkins:

  • go to the Manage Jenkins page, 
  • find the Script Console link in the Tools and Actions section, 
  • copy and paste the script into the text field 
  • run it

Saturday, May 01, 2021

ABCDE in Crux

After my OS update to Crux 3.6 (3.6.1 to be precise) I cleaned non-main (core, opt, xorg) packages. One of the side effects is that I lost abcde. It was removed from the contrib collection, because of inactive maintainer. Along abcde, the cd-discid was also deleted from the same reason. I decided to add them to my port collections (https://wawrzek.name/crux/repo/). I started from the old contrib ports. Looking at sources I noticed that there is a recent patch for cd-discid. I included it into my port. I also encounter problems in running abcde with my config. There were missing musicbrainz Perl modules, so I added ports for them as well.

Monday, March 01, 2021

Pulse and default sound card

For some reason PulseAudio wants to send sounds from my computer to the HDMI monitor, with rather crappy speakers, rather than to my headphones.

To stop it I set up a default output by editing /etc/pulse/default.pa. In my case the right configuration was:

set-default-sink alsa_output.pci-0000_38_00.6.analog-stereo

Saturday, November 28, 2020

Enforce module load in CRUX

At the moment, my Linux (CRUX 3.5) does not load a kernel module for the my mainboard monitoring chipset. The mainboard is GigaByte GA-M68MT-S2 and the chip is Nuvoton NCT6775F. I need data from that chip for Conky or LM Sensors. To ensure it's available I've modified the /etc/rc.modules file, by adding

modprobe nct6775

Friday, November 20, 2020

Debugging of Ansible and Molecule

As I descrbided in this article additional Ansible options are passed at the end of molecule command. This can be use to increase verbosity of Ansible:
molecule converge -- -vvvv
Molecule own debug information can be printed to the standard output with *--debug** option, but it has be specify without molecule command. For example:
molecule --debug converge
Of course, both can be mixed togther:
molecule --debug converge -- -vvvv

Sunday, November 01, 2020

edsn: Elite Dangerous Star Neighborhood

 Description

Virtual, for ever I was fascinating with maps of our star neighbourhood. Just like the one from European Southern Observatory website. Recently, I've enjoyed playing Elite Dangerous. I enjoy it even more, because it works perfectly fine on Linux (with Proton).  One of the great thing about Elite is freedom to roam between stars and visit, so many star neighbourhoods. 

The amount of stars is breathtaking. It's hard to visualize the ones which are close to the system you are in. So, I wrote a small Python script to get data from edsm and prepare them to be visualized with GNUplot.

Examples

Sol

Home, sweet home.

Stars in less than 15 Light Years from Sol

These are the commands to produce the SVG output after loading script and data into Gnuplot.

set term svg size 1600,1200
set view 45, 290, 1.25, 1.5
set output 'sol-r15.svg'; replot

Achenar

 I didn't know that the space around Achenar is so empty (OK. I don't have permit yet).

Stars in less than 15 Light Years from Achenar

These are the commands to produce the SVG output after loading script and data into Gnuplot.

set term svg size 1600,1200
set view 45, 275, 1.25, 1.5
set output 'achenar-r15.svg'; replot

Thursday, October 22, 2020

Kernel module info

A few commands to help in checking and adjusting kernel module options: 

  1. Display information about kernel module, including all options

    modinfo $KERNEL_MODULE

  2. Display information about current value for each option

    for i (/sys/module/$KERNEL_MODULE/parameters/*) {\
      echo $(basename $i);\
      cat $i\
    }


  3. Change kernel module option without restart (temporary)

    echo "$NEW_VALUE" > /sys/module/$KERNEL_MODULE/parameters/$OPTION

  4. Load module with an option set to value
    insmod $PATH_TO_MODULE/$KERNEL_MODULE.ko\ $OPTION1=$VALUE1 \
    $OPTION2=$VALUE2


  5. Ensure kernel module option is set during boot up with grub
    1. Open /etc/default/grub
    2. Add/edit following line
      GRUB_CMDLINE_LINUX='$KERNEL_MODULE.$OPTION=$VALUE'

Of course $KERNEL_MODULE, as well as all other strings with "$" at the beginning, has to be replaced by appropriate value.

In CRUX /etc/default/grub does not exist, so it has to be created.

 Useful links

Sunday, October 11, 2020

Shell script to setup system with new kernel

 

This is a small script helping in setting up a new kernel. It copies files from the Linux kernel source directory and builds the matching initramfs file.

 

#!/bin/sh
#
set -x
if [ $(dirname $PWD) != '/usr/src' ]
then
	echo ""
	echo "Please change directory to a one with Linux kernel sources"
	exit 1
fi

version=$(basename $PWD |awk -F\- '{print $2}')


ls boot/vmlinuz-${version}-* 2> /dev/null && \
	build=$(ls -1 /boot/vmlinuz-${version}-*| sort -n | awk -F\- '{print $3}') || \
	build=1

release=${version}-${build}
cp System.map /boot/System.map-${release}
cp arch/x86/boot/bzImage /boot/vmlinuz-${release}
mkinitrd /boot/initramfs-${release}.img ${version}

grub-mkconfig > /boot/grub/grub.cfg

Friday, August 28, 2020

Molecule and extra variables for Ansible

Imagine that you want to use Molecule to test an Ansible role with a not default value. That's just one off test, so adjusting molecule configuration in the converge.yml (or similar) file is not required. It's not very prominent in documentation, but you can pass Ansible variables after "--" (minus, minus) in the molecule commend. For example:

molecule converge -- -e "role_new_var=true"

Wednesday, August 26, 2020

How to find a word in files spread over many subdirectories

Imagine that you need to try to find a word in hundreds files located in a plenty of subfolders. grep -Rc might be a good candidate, but it's going to print the list of all files and the word is in just a handful of them. To make the output better grep can be joined with awk (and in the example below with zsh). 

I needed to find an example of sqlcmd command usage in my Ansible roles collection. I used the power of ZSH globbing (a few other examples can be found here and in other places) to limit search to yml file only. ZSH "**" pattern is also a nice replacement to the grep "--recursive"option. In awk the "-F" flag replaces the standard delimiter with the ":" character and prints only lines with second field not equal to zero.

grep -c sqlcmd **/*.yml | awk -F":" ' ($2 != 0 ){print $0}'

Wednesday, April 29, 2020

Get Public IP address of Azure VM from shell on VM - part II

Last year I wrote a quick note with a method of getting Azure VM public address from the shell on VM. Recently I spent a bit more time to look at https://docs.microsoft.com/en-us/azure/virtual-machines/linux/instance-metadata-service and found more precise call to get IP address from the metadata url: 

curl -H Metadata:true \
 "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-08-01&format=text"

BTW. The same method allows to get a private IP address.

curl -H Metadata:true \
 "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2017-08-01&format=text"

Tuesday, April 07, 2020

Painful encounter with oversized Postgres (9.6)

I had not so nice encounter with an oversize PostgresDB. Somehow rather small DB ballooned to 256 GB (what was a partition size). I'm not PostgresDB expert. I can barely use it, but I've spent some time doing internet search and that might be useful for other (or for me in the future).

In the end I regained nearly 200 GB of space with simple command:

vacuum full

As it is suggested by this Stack Overflow question.

The key to find above suggestion was to understand that the problem was related to pg_toast table. I found that fact out using oid2name tool, which BTW in Ubuntu is in /usr/lib/postgresql/10/bin/.


/usr/lib/postgresql/10/bin/oid2name -H localhost -U postgres -d postgres -f 24806

From database "postgres":
  Filenode      Table Name
--------------------------
     24806  pg_toast_24729


I learnt about the tool from this document and learnt how to use from the man page.

It's also worth to remember that following query didn't help me. I think it's nice, so posting it here, anyway:

SELECT table_schema, 
    table_name, 
    pg_relation_filepath('"'||table_schema||'"."'||table_name||'"') 
FROM information_schema.tables 
WHERE 
    pg_relation_filepath('"'||table_schema||'"."'||table_name||'"') 
LIKE 'base/12404/248%';

The above query was built based on another good suggestion from Stack Overflow. This one shows all big tables.

SELECT
    table_schema,
    table_name,
    pg_relation_size('"'||table_schema||'"."'||table_name||'"')
FROM  information_schema.tables
ORDER BY 3
LIMIT 10

Monday, March 16, 2020

prt-get/pkgmk adjustments

If I'm ever going to lose my current Crux config. I need to remember, to adjust following files for Crux:
  • /etc/prt-get.conf to ensure that scripts as executed:

    runscripts yes            # (no|yes)


  • /etc/pkgmk.conf to ensure that md5sums are ignored:

    PKGMK_IGNORE_MD5SUM="yes"