Akom's Tech Ruminations

Various tech outbursts - code and solutions to practical problems

Linux Jenkins Swarm Slaves on Windows using Puppet

Posted by Admin • Monday, October 12. 2015 • Category: DevOps, Linux
Jenkins Swarm plugin is great, and instrumentation for Linux is fairly well-known, but what about Windows? Here is one approach for setting it up as a windows sevice (what we want Puppet to do):
  1. Download Swarm jar
  2. Download winsw from Kohsuke
  3. Rename winsw to jenkins_swarm.exe
  4. Create jenkins_swarm.xml
  5. Run jenkins_swarm.exe install
  6. Start service
Now for the details

Continue reading "Jenkins Swarm Slaves on Windows using Puppet"

Linux Jenkins Windows Slaves cannot install JDK

Posted by Admin • Monday, October 12. 2015 • Category: DevOps, Linux
Despite making the jenkins slave user a local administrator. The slave runs tools\hudson.model.JDK\JDK_1.7\jdk.exe (or 1.6), and fails (see the extended post body for the log).

If you're seeing Error 1722.There is a problem with this Windows Installer package, then most likely you have another (newer?) version of this JDK installed system-wide. Uninstall from Control Panel, try again.

Continue reading "Jenkins Windows Slaves cannot install JDK"

Linux Webex in Ubuntu 15.04

Posted by Admin • Monday, September 28. 2015 • Category: Linux
The following was required to get shared screens to display:
  1. Using openjdk 7 (although I have 7 and 8 installed, 8 is default, and I removed oracle java - but the browser plugin still runs 7)
  2. Follow Option 1 from http://askubuntu.com/a/623397/238077
  3. If you are missing libxmu6, install the 32 bit version: sudo apt-get install libxmu6:i386 (this is actually mentioned in the answer, but I missed it)
  4. Firefox should now support webex

Linux Getting VMware vSphere 5.5 Server Console to work on Ubuntu Linux 15.04

Posted by Admin • Wednesday, September 16. 2015 • Category: DevOps, Linux
As everyone who uses this product eventually finds out, VMware vSphere (5.5) browser-based web client will generally work in Linux (at least in chrome), but opening a server console is nearly impossible. The web client requires flash 11.5, and of course there is no linux flash above 11.2...

After a lot of dancing I managed to get it working in Firefox 40.0.3 on Ubuntu 15.04. Here are are the steps in a nutshell:
  1. Clear out your ~/.mozilla ... True, you may not need to, but I did. Start firefox once, then exit.
  2. Install pipelight
  3. Enable flash in pipelight (I wound up doing this globally and for current user, not sure which one did it
  4. Install the VMware client integration plugin (if you did before, uninstall and reinstall). That's the download you get on the bottom of the vSphere login page
  5. Start firefox and see if it works. If not, fiddle with the plugins (under extensions). You may see two versions of flash. I was not able to disable 11.2 from the UI without also disabling version 18 from pipelight - so I wound up deleting /usr/lib/mozilla/plugins/flashplugin-alternative.so (but you can also apt-get remove flashplugin-installer)
  6. Explicitly go to extensions again and switch all VMware items to "Always Activate"
  7. Restart firefox over and over throughout the process :-)


Good luck



Update: After some system updates, the pipelight plugins stopped appearing in Firefox, and needless to say nothing worked. Deleting ~/.mozilla again seemed to resolve it (You then have to switch all the plugins to "Always Activate" again).

Linux Maven toolchains and puppet

Posted by Admin • Monday, August 31. 2015 • Category: DevOps, Linux
For today's trick we'll be setting up maven toolchains.xml files on Jenkins slaves that are managed by puppet. One option for doing this is to use the Jenkins Config File Provider plugin - but all that really does is push a predetermined file to the slaves before running the job (now if it only worked together with the Jenkins tool provider data...) In our case, we have Jenkins slaves with different OS versions and thus different toolchain locations - for example, minor JDK releases differ across OS's and thus the paths are not the same.

Instead, the plan is to do two things:
  1. Create a puppet custom fact with installed tool info
  2. Use it from a puppet template
Here is how

Continue reading "Maven toolchains and puppet"

Linux Getting ETVnet.com to play video on Ubuntu

Posted by Admin • Friday, June 12. 2015 • Category: Linux
In Ubuntu 15.04 it already works in Firefox as long as you have vlc installed, and you select /usr/bin/vlc the first time Firefox asks you what to do. Not so much in Chrome... Chrome uses xdg-open to determine what to launch. By default, xdg-open has no idea what to do with a mms://1.2.3.4/something.wmv style URL, so we need to set up a protocol action in xdg, as per http://askubuntu.com/questions/190895/how-to-change-what-xdg-open-does-with-ssh-userip-liniks :
xdg-mime default vlc.desktop x-scheme-handler/mms


UPDATE: I find that vlc does not gracefully recover from eTVnet's "glitches" - sound stops playing but video continues. Surprisingly, Totem does not have this issue, so I swtiched:
xdg-mime default totem.desktop x-scheme-handler/mms

Linux Simple puppet update-alternatives

Posted by Admin • Thursday, June 11. 2015 • Category: DevOps, Linux
This is a quick and dirty interface to update-alternatives on Centos/Redhat/Ubuntu for puppet. Seems to work well and doesn't require any modules.
Usage example: alternatives_update { 'java': versiongrep => '1.8' }

  # Manipulates alternatives using update-alternatives.
  # Supports RHEL, Centos and Suse.
  # Ubuntu not tested (yet).
  #
  # There is rudimentary alternatives support in the java class,
  # but it's rather limited and doesn't support most platforms and java versions.
  define alternatives_update (
    $item = $title,   # the item to manage, ie "java"
    $versiongrep,     # string to pass to grep to select an alternative, ie '1.8'
    $optional = true,  # if false, execution will fail if the version is not found
    $altcmd   = 'update-alternatives' # command to use
  ) {

    if ! $optional {
      # verify that we have exactly 1 matching alternatives, unless it's optional
      exec { "check alternatives for ${item}":
        path    => ['/sbin','/bin','/usr/bin','/usr/sbin'],
        command => "echo Alternative for ${item} version containing ${versiongrep} was not found, or multiple found ; false",
        unless  => "test $(${altcmd} --display ${item} | grep '^/' | grep -w -- $versiongrep | wc -l) -eq 1",
        before  => Exec["update alternatives for ${item} to ${versiongrep}"],
      }
    }

    # Runs the update alternatives command
    #  - unless it reports that it's already set to that version
    #  - unless that version is not found via grep
    exec { "update alternatives for ${item} to ${versiongrep}":
      path    => ['/sbin','/bin','/usr/bin','/usr/sbin'],
      command => "${altcmd} --set ${item} $( ${altcmd} --display ${item} | grep '^/' | grep -w -- $versiongrep | sed 's/ .*$//' ) ",
      unless  => "${altcmd} --display ${item} | grep 'currently points' | grep -w -- $versiongrep ",
      onlyif  => "${altcmd} --display ${item} | grep '^/' | grep -w -- $versiongrep", # check that there is one (if optional and not found, this won't run)
    }

  }
 

Linux Getting a systemd unit to read your .bashrc file for its environment

Posted by Admin • Thursday, June 11. 2015 • Category: Linux
Although it'd be nice to have all of your services not rely on their shell environment, sometimes it is unavoidable. In my case, some of the systems are still on init.d (redhat <=65, centos <=65, etc), but some are on systemd (7.0's, suse 12, etc). The old init scripts rely on the .bashrc file. In order to make the service run consistently on both flavors, I had to teach systemd to read the user's bashrc file. This is what it looks like.


[Service]
Type=simple
User=some_user
# Process the normal environment files for this user by starting a login shell
# and outputting it all in a temp file.  This makes it compatible
# with the non-systemd init scripts that still rely on .bashrc
ExecStartPre=/bin/bash --login -c 'env > /tmp/.magic-environment-file'
ExecStart=SERVICE_COMMAND
EnvironmentFile=-/tmp/.magic-environment-file
 
The "-" before the filename seems to make systemd ignore the error if the file is not there (for example, on the first run of the pre command). Now, obviously, you could just set your ~some-user/.bashrc as the EnvironmentFile - but your mileage will vary. systemd will not interpolate variables and it will ignore lines starting with "export"...

Code and Hacks QOS settings for pfSense - optimizing Google Hangouts and SIP

Posted by Admin • Monday, June 8. 2015 • Category: Code and Hacks
pfSense router configuration for Traffic Shaping is relatively convoluted if you're not already familiar with the interface. Assuming that you've used one of the wizards to create the default queues (default, ack, voip, p2p), what you have to do is:
  1. Set the upload bandwidth (Traffic Shaper->By Interface->WAN). This may be optional if using the default PRIQ mode
  2. Create a rule to match Google Hangouts traffic. According to this helpful post, the UDP ports hangouts uses are 19302-19309 (I'm assuming that outbound UDP is open). Therefore we create a new "match" type firewall rule with these ports as destination and assign them to the voip queue. Here is an example of a Floating rule to do that:
    Example Floating Rule
  3. For SIP (if you have SIP devices or softphones), you can make a Layer 7 rule (Traffic Shaper -> Layer 7) that assigns sip traffic to the voip queue
This is, of course, just a note for myself.

Linux Wifi doesn't work after resume from suspend in Ubuntu 15.04 (Dell Latitude E4310)

Posted by Admin • Sunday, May 17. 2015 • Category: Linux
Although toggling the hardware Wi-Fi switch usually fixed this issue, it was decidedly annoying for the non-technical user of this laptop. Googling for a well-known solution turned up several, but none of these worked. I found that either running
nmcli r wifi off
nmcli r wifi on
usually worked, so I tried to stick that into /etc/pm/sleep.d - but that script was never used. I then discovered that in 15.04 Ubuntu switched to systemd, which requires a service file in /etc/systemd/system/ In the end, it turns out that the problem is intermittent, therefore it is not possible to simply toggle networking blindly and hope for the best. Half the time it will not work. The ultimate solution turned out to be a combination of a toggle and a check, and both could be done via network manager, as follows:

Continue reading "Wifi doesn't work after resume from suspend in Ubuntu 15.04 (Dell Latitude E4310)"

Hardware Hacks Increasing capacity of the PetSafe Simply Clean Litter box

Posted by Admin • Tuesday, March 10. 2015 • Category: Hardware Hacks
Although I wrote a somewhat harsh review of the PetSafe Simply Clean Litterbox on Amazon, I still felt that there was some potential to this thing. What I wanted most from it was a very low maintenance experience, and having to empty the waste bin every 3 days simply didn't seem to qualify. When I go on vacation (and please don't start a neglect flame war over this) I leave the cats alone - monitored by 4 cameras, auto feeder, 5 gallon water dispenser and many litter boxes. I don't want to bother relatives with the long drive to come over just to clean cat poo, and my cats do not take well to being moved. Or to strangers. Or to basically anything besides the quiet at-home life that is exactly like yesterday.

Therefore, what I'm looking for is a larger waste bin. The litterbox doesn't handle capacity problems well - it backs up, jamming up against the chute cover, making a mess and eventually just shutting itself off. What I need to do is to create a jam-resistant path for the waste to go, somewhat like this patent here. The idea is good but it seems a bit too complex for my taste, and of course it is intended for a rectangular raking box like the LitterMaid. So, I chose to do the simplest and most reliable thing I could think of.

Continue reading "Increasing capacity of the PetSafe Simply Clean Litter box"

Linux Building RPMs for an older version of CentOS

Posted by Admin • Thursday, November 20. 2014 • Category: Linux
If you build RPMs on Centos 6.X (6.5 in my case) and then try to install them on Centos 5.X (5.10 in my case), bad things happen. Ironically, bad things happen even though my RPM contains a single jar file, and is thus entirely platform independent. Here is what I see:
Running rpm_check_debug
ERROR with rpm_check_debug vs depsolve:
rpmlib(FileDigests) is needed by my-rpm-1.0.x86_64
rpmlib(PayloadIsXz) is needed by my-rpm-1.0.x86_64
If you google for a solution, most people suggest running a virtual with Centos 5 just so you can build the RPMs, but this is apparently not necessary in this simple case, as you can simply specify a few flags. Basically, Centos 5 can't decompress the default archive format, and doesn't support the new digest algorithm.

Since I am using maven-rpm-plugin to build RPMs, my modifications look as follows:

                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>rpm-maven-plugin</artifactId>
                    <version>2.1-alpha-3</version>
                    <configuration>

                      <!-- skipping irrelevant items -->

                        <defineStatements>
                            <!-- don't strip jar files, it takes forever and is useless -->
                            <defineStatement>__os_install_post %{nil}</defineStatement>
                            <!-- for Centos6 -> Centos5 forwards compatibility -->
                            <defineStatement>_source_filedigest_algorithm md5</defineStatement>
                            <defineStatement>_binary_filedigest_algorithm md5</defineStatement>
                            <defineStatement>_source_payload w9.bzdio</defineStatement>
                            <defineStatement>_binary_payload w9.bzdio</defineStatement>
                        </defineStatements>

                   </configuration>
                </plugin>
 
If you're using fpm or rpmbuild, you can just take these lines and make them %define's in your spec.

Linux Backing up cPanel 11 hosted account with wget and dav/rsync

Posted by Admin • Thursday, August 21. 2014 • Category: Linux
I want to back up my hosting account regularly by retrieving everything onto my box somewhere else (my home server). By regularly, I mean every day. I want this to happen automatically. cPanel makes that hard to do, but there is always a way to script things.

First, let's break down what we want to back up:
  1. Files
  2. Databases and email forwarders
Files are fairly easy. Obviously we don't want to use the full backup functionality of cPanel because we'd be transferring your entire storage space each and every time, even if nothing has changed. In order to do it efficiently, you have a few options:
  1. FTP account using recursive wget (create an FTP account with required access and teach an ftp client of your choice to recursively transfer everything. Hopefully this client skips unchanged files)
  2. WebDAV using rsync (this is what I'm using). Just mount, back up, unmount
Databases and Email stuff is not as easy, as we do have to log into cPanel. The trick to logging in to cPanel is:
  1. Submit your login to the log-in page and save cookies
  2. Parse the resulting file, find the backup link which includes your session name in the URL and hit that
  3. Only accept .gz files (DB backups and email stuff), but avoid hitting /logout, and don't start spidering the entire website
So, how do we do this? Here is the plan

Continue reading "Backing up cPanel 11 hosted account with wget and dav/rsync"

Bash completion for tmux multi-window like cssh/mssh

Posted by Admin • Monday, May 19. 2014
Although I normally use mssh for my multi-window SSH client, sometimes I work through an non-graphical SSH connection, and tmux really comes in handy. I've been using the ssh-multi script by D.Kovalov, but since I already have "clusters" files for cssh and mssh, I figured that it should be more convenient to use them by typing aliases. The script below is the result. )

This goes into /etc/bash_completion.d/tmuxmulti

Hope it helps

Continue reading "Bash completion for tmux multi-window like cssh/mssh"

Linux mssh Bash completion

Posted by Admin • Friday, February 28. 2014 • Category: Linux
Clusterssh (cssh) is great, but I was getting a little fed up with unmanageable terminal windows. They either go all over my monitors or get lost, and they are hard to move and resize.

So I switched to mssh, which solves all that because all the terminals are in one window.... but, it doesn't read my /etc/clusters file! In fact, nobody seemed to even know what file it does read.

What to do? First of all, for the record, it reads its aliases (-a) from ~/.mssh_clusters This file is exactly like /etc/clusters, except for a colon, like so:
alias1: host host host
alias2: host host host

Now that we got that straight, let's make a bash completion file

Continue reading "mssh Bash completion "