Jenkins and Splunk (Cloud)

There is the Jenkins Plugin to set log export to Splunk. In Splunk Marketplace you can find the Splunk App for Jenkins, but the app works only for Splunk Enterprise, not the Cloud edition. Documentation describe how to connect Jenkins to Splunk. It includes values for cloud deployment as well as on-premises one. Thanks to that, the connection was easy to establish, but there were no logs from Jenkins. I had to manually created 4 indexes (I got them from this discussion) to get Jenkins logs visible in Splunk.

Sims 4 (and Origin client) on Linux with Steam

Recently, I've spent some time trying to make Sims 4 running on Linux. (That's the way you might spend time when you have a growing up daughter.) The ProtonDB entry was (and still is) GOLD, so it was encouraging. However, it didn't work on the first computer I tried. Steam client showed the game ran, but there was no window. Neither for the Sims 4 game, nor for the Origin client. It was:

• Distro: Mint 20.2
  
• CPU: Intel i5-7400
• GPU: Nvidia (Zotec GT 610 1GB)
• Kernel: 5.4.0-88 (Ubuntu/Mint)
• Drivers: Nvidia closed source 390.144
• Mesa: 21.0.3
• Proton: various versions
• Sims 4 works: No
  

Then I tried on my Crux machine, and it worked fine. I don't remember if I had to add PROTON_USE_WINED3D=1. The machine had:

• Distro: CRUX 3.6.1
  
• CPU: AMD Ryzen 5 2400G
• GPU: AMD RX 6600 XT
• Kernel: 5.13.2
• Drivers: amdgpu
• Mesa: 21.2.2
• Proton: 6.3.7
• Sims 4 works: Yes
  

So I thought that maybe GPU from the original  machine was too old, or NVidia drivers were causing problems. I replace the Nvidia GPU an AMD FirePro V5900 card. It didn't help.

The next step was to run a test on my old Dell XPS-13 laptop. I installed the OpenSuse Leap 15.3 on it. I played a bit with Proton version, but in the end Sims started with the Proton 7.0rc2-GE-1, the latest release from the Glorious Eggroll branch.

• Distro: OpenSuse Leap 15.3
  
• CPU: Intel i7-4510
  
• GPU:Intel i915
  
• Kernel: 5.13.18-59.10-default
• Drivers: intel
• Mesa: 20.2.4
• Proton: 7.0rc2-GE-1
• Sims 4 works: Yes
  

In the same time, I realized that I could swap CPUs between machines. The Ryzen had the build in Vega GPU core, which was not in use. The first phase was to check that Sims 4 could start on CRUX without discrete GPU. It worked fine. Then I swapped the CPUs (with motherboards). The CRUX (with Intel CPU and AMD GPU) worked fine, but the Mint with AMD Ryzen (and integrated AMD GPU) still struggled. I tried using the Proton from GE branch, which helped me on OpenSuse. No luck.

• Distro: Mint 20.2
  
• CPU: AMD Ryzen 5 2400G
• GPU: AMD Radeon Vega 11
• Kernel: 5.15.6-1-default
  
• Drivers: amdgpu
• Mesa:21.3.1
• Proton: 6.3.8
• Sims 4 works: No

I started to consider that something wrong is with Mint. The Sims 4 problem seems to be the problem with Origin client.  The first relatively easy change was to use the KDE instead of Cinnamon as a Window Manager for the Mint system. It didn't help. Next was to install OpenSuse along Mint. The only change compared to the Dell laptop was to use the Tumbleweed rather than the Leap edition.

• Distro: OpenSuse Tumbleweed
  
• CPU: AMD Ryzen 5 2400G
• GPU: AMD Radeon Vega 11
• Kernel: 5.15.6-1-default
  
• Drivers: amdgpu
• Mesa:21.3.1
• Proton: 6.3.8
• Sims 4 works: Yes
  

This time it worked fine. No issues. In conclusion, there is something wrong with Mint, but Sims 4 and Origin client works fine on Linux with Steam.

dmidecode - the command I always forget about

 

From time to time I need to check some details of the hardware in one of my Linux server. There is a good command to do this, which I know exists and have good functionality, but cannot remember the actual text to call it. The command in question is:

dmidecode

And below links with examples and explanation how to use it:

•  https://www.ubuntupit.com/simple-and-useful-dmidecode-commands-for-linux/
•  https://linuxiac.com/dmidecode-get-system-hardware-information-on-linux/

How to check Jenkins credentails

If you every need to check what are the actual password in Jenkins credentials check this small groovy script  by Tim Jacomb. I helped me to confirm that credentials were corrupted during saving and restoring with Configuration as a Code plugin.

To use it (and any other groovy script). In your Jenkins:

• go to the Manage Jenkins page, 
• find the Script Console link in the Tools and Actions section, 
• copy and paste the script into the text field 
• run it
  

ABCDE in Crux

After my OS update to Crux 3.6 (3.6.1 to be precise) I cleaned non-main (core, opt, xorg) packages. One of the side effects is that I lost abcde. It was removed from the contrib collection, because of inactive maintainer. Along abcde, the cd-discid was also deleted from the same reason. I decided to add them to my port collections (https://wawrzek.name/crux/repo/). I started from the old contrib ports. Looking at sources I noticed that there is a recent patch for cd-discid. I included it into my port. I also encounter problems in running abcde with my config. There were missing musicbrainz Perl modules, so I added ports for them as well.

Pulse and default sound card

For some reason PulseAudio wants to send sounds from my computer to the HDMI monitor, with rather crappy speakers, rather than to my headphones.

To stop it I set up a default output by editing /etc/pulse/default.pa. In my case the right configuration was:

set-default-sink alsa_output.pci-0000_38_00.6.analog-stereo

Enforce module load in CRUX

At the moment, my Linux (CRUX 3.5) does not load a kernel module for the my mainboard monitoring chipset. The mainboard is GigaByte GA-M68MT-S2 and the chip is Nuvoton NCT6775F. I need data from that chip for Conky or LM Sensors. To ensure it's available I've modified the /etc/rc.modules file, by adding

modprobe nct6775

Debugging of Ansible and Molecule

As I descrbided in this article additional Ansible options are passed at the end of molecule command. This can be use to increase verbosity of Ansible:

molecule converge -- -vvvv

Molecule own debug information can be printed to the standard output with *--debug** option, but it has be specify without molecule command. For example:

molecule --debug converge

Of course, both can be mixed togther:

molecule --debug converge -- -vvvv

edsn: Elite Dangerous Star Neighborhood

 Description

Virtual, for ever I was fascinating with maps of our star neighbourhood. Just like the one from European Southern Observatory website. Recently, I've enjoyed playing Elite Dangerous. I enjoy it even more, because it works perfectly fine on Linux (with Proton).  One of the great thing about Elite is freedom to roam between stars and visit, so many star neighbourhoods. 

The amount of stars is breathtaking. It's hard to visualize the ones which are close to the system you are in. So, I wrote a small Python script to get data from edsm and prepare them to be visualized with GNUplot.

Examples

Sol

Home, sweet home.

These are the commands to produce the SVG output after loading script and data into Gnuplot.

set term svg size 1600,1200
set view 45, 290, 1.25, 1.5
set output 'sol-r15.svg'; replot

Achenar

 I didn't know that the space around Achenar is so empty (OK. I don't have permit yet).

These are the commands to produce the SVG output after loading script and data into Gnuplot.

set term svg size 1600,1200
set view 45, 275, 1.25, 1.5
set output 'achenar-r15.svg'; replot

Kernel module info

A few commands to help in checking and adjusting kernel module options: 

1. Display information about kernel module, including all options
   
   modinfo $KERNEL_MODULE
   
   
2. Display information about current value for each option
   
   for i (/sys/module/$KERNEL_MODULE/parameters/*) {\
     echo $(basename $i);\
     cat $i\
   }
   
   
3. Change kernel module option without restart (temporary)
   
   echo "$NEW_VALUE" > /sys/module/$KERNEL_MODULE/parameters/$OPTION
   
   
4. Load module with an option set to value
   insmod $PATH_TO_MODULE/$KERNEL_MODULE.ko\ $OPTION1=$VALUE1 \
   $OPTION2=$VALUE2
   
   
5. Ensure kernel module option is set during boot up with grub
1. Open /etc/default/grub
2. Add/edit following line
   GRUB_CMDLINE_LINUX='$KERNEL_MODULE.$OPTION=$VALUE'
   

Of course $KERNEL_MODULE, as well as all other strings with "$" at the beginning, has to be replaced by appropriate value.

In CRUX /etc/default/grub does not exist, so it has to be created.

 Useful links

• https://www.cyberciti.biz/tips/how-to-display-or-show-information-about-a-linux-kernel-module-or-drivers.html
• https://devarea.com/linux-kernel-development-kernel-module-parameters/
• https://www.howtoforge.com/tutorial/kernel-boot-parameter-edit/
• https://www.linuxsecrets.com/1455-adding-kernel-boot-parameters-using-grub-on-linux
  

Shell script to setup system with new kernel

 

This is a small script helping in setting up a new kernel. It copies files from the Linux kernel source directory and builds the matching initramfs file.

 

#!/bin/sh
#
set -x
if [ $(dirname $PWD) != '/usr/src' ]
then
	echo ""
	echo "Please change directory to a one with Linux kernel sources"
	exit 1
fi

version=$(basename $PWD |awk -F\- '{print $2}')


ls boot/vmlinuz-${version}-* 2> /dev/null && \
	build=$(ls -1 /boot/vmlinuz-${version}-*| sort -n | awk -F\- '{print $3}') || \
	build=1

release=${version}-${build}
cp System.map /boot/System.map-${release}
cp arch/x86/boot/bzImage /boot/vmlinuz-${release}
mkinitrd /boot/initramfs-${release}.img ${version}

grub-mkconfig > /boot/grub/grub.cfg

Molecule and extra variables for Ansible

Imagine that you want to use Molecule to test an Ansible role with a not default value. That's just one off test, so adjusting molecule configuration in the converge.yml (or similar) file is not required. It's not very prominent in documentation, but you can pass Ansible variables after "--" (minus, minus) in the molecule commend. For example:

molecule converge -- -e "role_new_var=true"

How to find a word in files spread over many subdirectories

Imagine that you need to try to find a word in hundreds files located in a plenty of subfolders. grep -Rc might be a good candidate, but it's going to print the list of all files and the word is in just a handful of them. To make the output better grep can be joined with awk (and in the example below with zsh). 

I needed to find an example of sqlcmd command usage in my Ansible roles collection. I used the power of ZSH globbing (a few other examples can be found here and in other places) to limit search to yml file only. ZSH "**" pattern is also a nice replacement to the grep "--recursive"option. In awk the "-F" flag replaces the standard delimiter with the ":" character and prints only lines with second field not equal to zero.

grep -c sqlcmd **/*.yml | awk -F":" ' ($2 != 0 ){print $0}'

Get Public IP address of Azure VM from shell on VM - part II

Last year I wrote a quick note with a method of getting Azure VM public address from the shell on VM. Recently I spent a bit more time to look at https://docs.microsoft.com/en-us/azure/virtual-machines/linux/instance-metadata-service and found more precise call to get IP address from the metadata url: 

curl -H Metadata:true \
 "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-08-01&format=text"

BTW. The same method allows to get a private IP address.

curl -H Metadata:true \
 "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2017-08-01&format=text"

Painful encounter with oversized Postgres (9.6)

I had not so nice encounter with an oversize PostgresDB. Somehow rather small DB ballooned to 256 GB (what was a partition size). I'm not PostgresDB expert. I can barely use it, but I've spent some time doing internet search and that might be useful for other (or for me in the future).

In the end I regained nearly 200 GB of space with simple command:

vacuum full

As it is suggested by this Stack Overflow question.

The key to find above suggestion was to understand that the problem was related to pg_toast table. I found that fact out using oid2name tool, which BTW in Ubuntu is in /usr/lib/postgresql/10/bin/.

/usr/lib/postgresql/10/bin/oid2name -H localhost -U postgres -d postgres -f 24806

From database "postgres":
  Filenode      Table Name
--------------------------
     24806  pg_toast_24729

I learnt about the tool from this document and learnt how to use from the man page.

It's also worth to remember that following query didn't help me. I think it's nice, so posting it here, anyway:

SELECT table_schema, 
    table_name, 
    pg_relation_filepath('"'||table_schema||'"."'||table_name||'"') 
FROM information_schema.tables 
WHERE 
    pg_relation_filepath('"'||table_schema||'"."'||table_name||'"') 
LIKE 'base/12404/248%';

The above query was built based on another good suggestion from Stack Overflow. This one shows all big tables.

SELECT
    table_schema,
    table_name,
    pg_relation_size('"'||table_schema||'"."'||table_name||'"')
FROM  information_schema.tables
ORDER BY 3
LIMIT 10

prt-get/pkgmk adjustments

If I'm ever going to lose my current Crux config. I need to remember, to adjust following files for Crux:

• /etc/prt-get.conf to ensure that scripts as executed:
  
  

  runscripts yes            # (no|yes)

  
  
  
• /etc/pkgmk.conf to ensure that md5sums are ignored:
  
  

  PKGMK_IGNORE_MD5SUM="yes"

  
  

Bash and Arrays

Short reminder for my 'extended memory' on how to deal with lists in bash.

To define:

• a list (an array): declare -a list
• a dictionary (an associative array): declare -A dict

To use an array in loop: for item in ${list[@]}

Now the best, at least for scripts testing:
To print a whole array: declare -p list (please note, no "$" before list name).


Found at https://www.tutorialkart.com/bash-shell-scripting/bash-array/

AMDGPU firmware and homebuild kernel

Not sure how it happens in less DIY type distributions, but in Crux I struggled to add firmware of AMDGPU drivers to a initrmfs. Maybe I haven't tried long enough, but without firmware in the initial ramfs, I could not properly start my desktop.

I learnt that drivers are on a disk, in what looked like the right directory (/lib/firmware/amdgpu/). They just were not loaded. After passing a few times the directory name via a CLI option I figured out that the path could be added to the dracut config.

Now, I have following config file on which make rebuilding kernel much easier.

cat /etc/dracut.conf.d/wawrzek.conf
install_items+=/lib/firmware/amdgpu/*

Thanks to that I can simple run command like that to add ramfs to:

mkinitrd /boot/initramfs-5.4.0-1.img 5.4.0

Copy files around in CentOS with SELinux (e.g. Nginx)

Imagine that you want to enable https traffic on a site served by Nginx. Sounds simple. A series of command like these should work:

scp example.com.* your_remote_server:
ssh your_remote_server
sudo mkdir /etc/nginx/ssl
sudo cp example.com.* /etc/nginx/ssl

Edit appropriate configuration file(s). Finally, run:

sudo systemctl restart nginx

and nothing, or rather an error message with information that nginx cannot access ssl certificate files. You check and files exists, so what's the problem?

The problem is a SELinux security context, diffrent for /etc/ngnix and /home/X

To change it you need to run chcon. For example:

chcon --reference /etc/nginx/nginx.conf /etc/nginx/ssl/example.com.*

More info at: https://www.cyberciti.biz/faq/rhel-centos-feora-linux-change-copy-selinux-context/

OpenSSL and certs

I was battling with SSL certificates recently and have two useful command I would like to store in my extended memory (this blog).


a) To ensure that all certificates in a bundle are OK. There should be a clear 'line of trust' in output.

FILENAME=your.domain.crt
openssl crl2pkcs7 -nocrl -certfile $FILENAME | openssl pkcs7 -print_certs -noout

b) To ensure that the EC (Eliptic Curve) key in the csr and the certifciate is equal to the actual signing key run these 3 commands. The public key for each command should be the same.

NAME=your.domain

openssl ec -pubout -in $NAME.key

openssl req -noout -pubkey -in $NAME.csr

openssl x509 -noout -pubkey -in $NAME.crt