Akom's Tech Ruminations

Various tech outbursts - code and solutions to practical problems

Linux PowerShell on Linux doesn't find modules when run through Puppet

Posted by Admin • Thursday, March 11. 2021 • Category: DevOps, Linux

I'm running PowerShell on Linux for the sake of using PowerCLI. An interesting thing happens: when pwsh is run interactively, in a terminal, the following works fine when PowerCLI is already installed:

pwsh -c 'Get-InstalledModule VMware.PowerCLI'

When puppet runs it - it doesn't find the module:

Get-Package: /opt/microsoft/powershell/7/Modules/PowerShellGet/PSModule.psm1:9445
Line |
9445 |          PackageManagement\Get-Package @PSBoundParameters | Microsoft. …
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | No match was found for the specified search criteria and
     | module names 'VMware.PowerCLI'.

Error: 'pwsh -c "if (-not(Get-InstalledModule VMware.PowerCLI)) { Exit 1 }"' returned 1 instead of one of [0]

After some experimentation, I narrowed it down to a single environment variable missing when puppet executes the command: HOME. Set HOME, and powershell will find modules. That gets us the following recipe:

  $my_user = 'some_user'

  exec { 'Install PowerCLI':
    path    => ['/bin', '/usr/bin'],
    command => 'pwsh -c \'Install-Module -Force:$true -Name VMware.PowerCLI\'',
    unless  => 'pwsh -c \'if (-not(Get-InstalledModule VMware.PowerCLI -ErrorAction silentlycontinue)) { Exit 1 }\'',
    user    => $my_user,
    environment => [ "HOME=/home/${my_user}"]
  }

Java Gradle Toolchains Support - different JVMs for compile and test

Posted by Admin • Thursday, February 4. 2021 • Category: DevOps, Java

I'm testing a product that needs to be compiled with JDK 8 but tested (sometimes) on JDK 11. This is now possible to do with maven surefire (although that took some effort). With gradle, I was doing it as follows, which is terrible, even if the path comes from configuration:


// The old way:
test {
    executable = '/some/hardcoded/path/to/java'
}
 

Now that Gradle 6.7+ has built-in toolchains support, it's trivial to configure the compile toolchain:


java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(11)
    }
}
 

But what about tests? What if I want to test with a different JVM? It's a little more verbose:


test {
    JavaToolchainService javaToolchainService = project.getExtensions().getByType(JavaToolchainService.class)
    def launcher = javaToolchainService.launcherFor{
        languageVersion = JavaLanguageVersion.of(11)
    }
    javaLauncher = launcher
    environment 'JAVA_HOME', launcher.get().metadata.installationPath //if your tests care about JAVA_HOME
}
 

Linux Puppet recipe for setting up autossh via systemd

Posted by Admin • Thursday, November 19. 2020 • Category: DevOps, Linux

I've always set up autossh in /etc/rc.local, but with CentOS 8 that doesn't work well (things start too early, etc). Luckily, there is a nicer way using systemd templates. Essentially, all you have to do is create one symlink and one config file per instance of autossh.

Example:

Make a config file


/etc/autossh/mything.conf:

OPTIONS=-N  -M 20000 -R8888:1.2.3.4:8888 5.6.7.8


Make a symlink

ln -s /usr/lib/systemd/system/autossh@.service /etc/systemd/system/autossh@mything.service


Test

systemctl start autossh@mything.service
journalctl -xe


But it's better to automate:

Puppet Recipe



define autossh(
    $service=$name,
    $args,
  ) {

    file {"/etc/systemd/system/autossh@${service}.service":
      ensure => link,
      target => '/usr/lib/systemd/system/autossh@.service'
    } ->
    file {"/etc/autossh/${service}.conf":
      content => "OPTIONS=${args}"
    } ->
    service {"autossh@${service}":
      ensure => running,
      enable => true,
    }

}

# and example usage - random port forwarding
autossh{'mything': args => '-N  -M 20000 -R8888:1.2.3.4:8888 host1'}
autossh{'mything2': args => '-N  -M 20000 -R8888:1.2.3.4:8888 host2'}
 

Using puppet to set Windows Computer Description

Posted by Admin • Monday, May 18. 2020 • Category: DevOps

My company's security department decided to assign Antivirus exclusion policies based on the value of the windows computer description. That means that I need to set computer descriptions ( net config server /srvcomment:"new description" ) to the same value on a whole bunch of windows machines. Doing that by hand is unappealing, and I already have puppet, so here is a simple solution:


MYCLASS/lib/facter/wincomputer_description.rb


# Gets computer decscription from 'net config server'
Facter.add("win_computer_description") do
  setcode do
    output = `net config server`
    output.split(/\n/).find{|it| it.start_with?('Server Comment') }.gsub(/Server Comment[\s]*/,'')
  end
end
 

MY_CLASS/manifests/init.pp


class MY_CLASS(
  $computer_description = 'Some-new-description',
) {
  if ($win_computer_description != $computer_description) {
    exec { 'Set computer description':
      path => ["C:\\windows\\system32"],
      command => "net config server /SRVCOMMENT:\"${computer_description}\"",
    }
  }
}
 

Low Tech Hacks Replacing Dishwasher Circulation Pump Bearings without buying a whole new unit

Posted by Admin • Friday, April 26. 2019 • Category: Low Tech Hacks

I have a DW80j3020us/AA Samsung dishwasher. It has been producing a loud noise for some time, and the noise kept getting worse. It was a matter of time before it either seized or melted the motor. The reason I know that the problem is with the circulation pump is that the noise is only audible while it's washing (circulating), not while draining or filling. Clearly the problem could be the motor (bearings) or some part of the impeller/grinder (less likely).

Samsung Dishwasher

My options were:

  1. New dishwasher
  2. New Circulation Pump motor DD31-00008A ($150)
  3. Try to change the bearings (I've found nothing online about anyone doing this)

You can find lots of videos online on how to properly replace the circulation pump motor/assembly. I followed the guides for removing the internal components down to the food chopper, since the chopper is on the motor shaft. This takes about 20 minutes. After that, it was easy - all I had to do was pull the bottom cover off and remove 4 screws to get the motor out. Then a magical thing happened.

Continue reading "Replacing Dishwasher Circulation Pump Bearings without buying a whole new unit"

Android doing Optimizing app 1 of 1 on every boot

Posted by Admin • Friday, May 4. 2018 • Category: Android

The message appears for a good 15 minutes every time my phone boots up. I followed the usual suggestions (wipe cache partition), and that didn't help



Figuring out what app is causing this issue is the hard part. I did it with logcat (I happened to have the Android Studio installed, so logcat is a tab, and it displays the log automatically as the plugged-in phone is booting). You don't need the whole thing, just the adb tools so you can run "adb logcat"



At the moment when the message appears, I saw this in the log:

05-04 14:48:10.388 1230-1230/? I/PackageManager.DexOptimizer: Running dexopt (dex2oat) on: /data/app/com.alltrails.alltrails-2/base.apk pkg=com.alltrails.alltrails isa=arm64 vmSafeMode=false debuggable=false oatDir = /data/app/com.alltrails.alltrails-2/oat

Well, now I know what to uninstall. I didn't need Alltrails anyway.

Disabling Windows Recycle Bin with Puppet on all versions of Windows

Posted by Admin • Thursday, March 22. 2018 • Category: DevOps

And when I say "All versions of Windows" I mean that I tested it on Server 2008, 2012 and 2016.



This was oddly hard to figure out, and most tutorials either apply to only one version of windows or to outdated tools. The best way to do this that I found was using Local Group Policy. Now, how to automate this? The puppet localgrouppolicy module didn't work at all when I tried it (and has not been updated since 2014). The proper way to do this is of course with a Domain-based Group Policy, but my machines are not members of a domain.



Fortunately, there is a new Microsoft tool called LGPO that allows for some degree of command-line control of the Local Group Policy. Download "LGPO.exe" here.

First, let's make a reusable policy text file that we can import on all machines:

  1. take a vanilla Windows machine that hasn't had any Group Policy customization, and use lgpo.exe to export the policy: "lgpo.exe /v /parse /u c:\windows\system32\GroupPolicy\User\Registry.pol" (at least that was appropriate in my case). You should get more or less empty output.
  2. Then use the Local Group Policy Editor to change "Do not move deleted files to the Recycle bin" (under User Configuration -> Administrative Tools -> All Settings) to "Enabled"
  3. Repeat step 1. You should see this one setting that you changed in the output. Redirect output to a file, this will be our text file
  4. You can test that importing this file will change the setting: "lgpo /r myfile.txt". (Change the setting back first, run this, then re-open the Local Group Policy Editor to see the change)

Now, we can set up puppet:

Continue reading "Disabling Windows Recycle Bin with Puppet on all versions of Windows"

Jenkins Pipeline: parallel and waitUntil, waiting until the other branch finishes

Posted by Admin • Monday, January 22. 2018 • Category: DevOps

Let's say that for the sake of speed, you are running two slow things in parallel:


parallel one: {
    node {
        sh "sleep 15"
    }
}, two: {
    node {
        //slow part:
        sh "sleep 10"  

        // now do something that needs "one" to finish.  There is a good chance that this will run too soon...
        sh 'wget http://something' //for example
    }
}
 

The problem, obviously, is that you can't be sure that the wget will run after part one finishes. (Let's assume that part one creates the file).

Continue reading "Jenkins Pipeline: parallel and waitUntil, waiting until the other branch finishes"

Code and Hacks Customizing Windows 7 keyboard and mouse to act like Linux

Posted by Admin • Friday, April 14. 2017 • Category: Code and Hacks

After using only Linux for nearly 20 years, I'm being forced to use a Windows machine for work. To make matters worse, I'm really (really) used to Enlightenment shortcuts, yet I use XFCE (Xubuntu). So, I'd like to have a seamless experience if I can help it. This is a log of my experimentation.



Note that I am not talking about a skin - I don't care that much about the looks, and I don't need the Windows desktop to look like Mint or Unity. What I want is for Windows to respond with the same level of fluidity that I get from Linux

Continue reading "Customizing Windows 7 keyboard and mouse to act like Linux"

Linux OCR on a large PDF using tesseract and pdftk

Posted by Admin • Thursday, January 19. 2017 • Category: Linux

This turned out to be harder than I thought. I found a large (50MB) PDF with about 50 pages, and none of the tesseract GUI's seemed to be able to handle it without crashing. The solution is to convert the PDF to TIFF so that command-line tesseract could handle it directly, but now ImageMagick couldn't handle that conversion as it was running out of memory (even with the limit settings). So the only option was to reduce the load on all the moving parts by splitting the PDF into pages.



Even after splitting the PDF and running each page through the PDF->TIFF->Tesseract->PDF chain I was still having issues:

Error in pixReadFromTiffStream: spp not in set {1,3,4}

Huh? So it turns out that sometimes you may wind up with an alpha channel in your TIFF and tesseract can't handle this. There is a solution, fortunately. So finally, I put all of these steps together into a script:

Continue reading "OCR on a large PDF using tesseract and pdftk"

Java Linux Running Jenkins Swarm client as a service via Upstart

Posted by Admin • Wednesday, December 21. 2016 • Category: DevOps, Java, Linux

This turned out to be fairly simple, with only one gotcha: do not follow the how-to's out there that tell you to use expect fork. The process doesn't technically fork. When I had that setting enabled, upstart commands would hang under very specific but repeatable conditions (if the process was killed externally).



So, here is my upstart conf file:

Continue reading "Running Jenkins Swarm client as a service via Upstart"

Linux Docker: Automatically remove containers that have been running too long

Posted by Admin • Thursday, October 20. 2016 • Category: DevOps, Linux

Why Because my Jenkins setup sometimes starts containers and forgets about them. Either it thinks it failed to start one, or the container itself has trouble starting. Either way, I'm left with containers that are running, trying to connect to Jenkins in vain, forever. The proper way to fix this is probably to have the containers timeout at some point, but that mechanism is broken.



Anyway, the fix I have is a true hack: find containers that have been up more than 2 days and kill them. None of our jobs should run for more than about a day, so this is a safe limit. Here is a bash script to do this:

Continue reading "Docker: Automatically remove containers that have been running too long"

Automotive Hardware Hacks Odometer Reprogramming - Nissan Sentra

Posted by Admin • Saturday, September 24. 2016 • Category: Automotive, Hardware Hacks

Disclaimer: It is perfectly legal to program an odometer to represent the correct mileage for the car.

Problem:

The car is a 2006 Nissan Sentra 1.8S. The instrument cluster is faulty and flips overdrive on and off at random. I ruled out the rest (switch, TCM). Thus, the cheapest option is to swap in another cluster, but that means that my mileage will be wrong. The replacement cluster I picked up on Ebay has 24K miles. Mine has 181K. Slight difference there.

Solution Summary:

Use an EZP2013 IC programmer to read the memory from your old cluster and write that to the new one. Clearly this assumes that you are still in possession of your old cluster and that its EEPROM is intact.

Solution Detail:

Continue reading "Odometer Reprogramming - Nissan Sentra"

Linux Jenkins command line ssh: Host key verification failed despite ssh-agent

Posted by Admin • Thursday, August 4. 2016 • Category: DevOps, Linux

After hours of "Why does it work locally but not in Jenkins", this error boils down to StrictHostKeyChecking... In other words, since the job runs as a user on a random slave, and this user most likely doesn't have a known hosts file with an entry for the target system, this fails rather cryptically. You think that the user's keys don't work, but that's not the problem.



The whole setup boils down to :

  1. Install ssh-agent plugin
  2. Configure credentials with a valid ssh key for your target
  3. Enable ssh-agent with that credential entry in your job config
  4. Add StrictHostKeyChecking=no to whatever ssh command you are using. Some examples:
    • GIT:
      export GIT_SSH_COMMAND="ssh -oStrictHostKeyChecking=no"
    • SSH:
      ssh -oStrictHostKeyChecking=no ....

Low Tech Hacks Replacing batteries in a Philips Norelco T980 Turbo Vacuum Trimmer

Posted by Admin • Wednesday, July 27. 2016 • Category: Low Tech Hacks

I bought this trimmer in 2008 and the NiCd batteries finally died in 2016 - not bad. Still, no reason to trash a working trimmer and contribute to the landfill problem. As it turns out, it's fairly easy to disassemble, and even easier to reassemble. So here are the steps, which probably apply to many similar models.

Continue reading "Replacing batteries in a Philips Norelco T980 Turbo Vacuum Trimmer"