Auto-Mounting VirtualBox Shared Folders with a Linux Guest OS

VirtualBox has a feature where a folder on the host machine can be shared with a guest VM. VirtualBox can also mount these shared folders automatically when the user logs into the guest OS. For Linux guests, these auto-mounted shared folders are mounted into the /media directory, along with the prefix sf_. For example, the shared folder sharedfiles will be mounted to /media/sf_sharedfiles.

As only the user group vboxsf is granted permissions to the auto-mounted shared folders, you’ll need to add your user to the group to access the shared folder. Otherwise you’ll get a “Permission denied” error when trying to navigate to the shared folder.

sudo gpasswd -a username vboxsf

Make sure you log out of the guest OS, then log in again for the group membership to take effect.

History of Christianity

Continuing the theme of history-based studies, the latest youth group study I led was on the History of Christianity. There is an incredible amount of literature available online, and we only covered a small fraction of events from Early Christianity and investigating the major groups of denominations. We also had a look at how Christianity and the churches in India formed particularly with their interaction with the western churches, and this was personally fascinating. Another group member covered the Crusades, while the Reformation will be covered in our next study. Here are the slides I used and make sure you have a look at the notes section for the talking points. I did use a special font for the title slide if it looks strange.

Intro to Data Modelling

When I was working for the Information and Data Management practice at SMS Management & Technology a few years back, I was asked to run a workshop for the Business Analysis Focus Group on data modelling. I put together an introductory presentation on data modelling targeted towards business analysts, which was a refresher on terminology and approaches. At the end of the presentation, I paired up everyone in the audience for two data modelling exercises: one person would act as a client while the other would act as a data analyst, eliciting data requirements and identifying a conceptual/logical data model. There were two scenarios so each person had an opportunity at the data analyst role, and both scenarios were based on real world projects I had previously worked on.

Download presentation
Download scenarios

Using Rails’ descendants in development

Rails provides a nice method descendants that returns all subclasses for a specified class. However, config.cache_classes = false is a default setting in development.rb (for the ability to reload classes on the fly) and as this tells Rails not to load classes until they are referenced, then calling descendants will generally return an empty array not the expected subclasses.

To be able to use descendants in development, you can add the following code to development.rb:

config.eager_load_paths += Dir['path/to/files/*.rb']
ActionDispatch::Reloader.to_prepare do
  Dir['path/to/files/*.rb'].each {|file| require_dependency file}
end

The first line tells Rails to load these particular files when Rails first starts up. The rest of the code tells Rails to require these particular files (including any new files/classes) on each request.

Note: if your path/to/files isn’t one of the standard directories that Rails watches (to trigger the reloader), you will need to add it to config.watchable_dirs:

config.watchable_dirs['path/to/files/'] = [:rb]

Islam

I decided to do a series of studies for our church youth group on various religions and started with Islam as Muslims form the second largest religious group globally. The intent was to cover the fundamentals of the Islamic faith, common terms that people may have come across, a brief history, and Islam’s relation to India, Christianity, and our local city. I’ve uploaded the notes I used.

Malayalam TV

Like most of the older folk in my ethnic community, my grandparents enjoy watching Malayalam television programming. They had jumped on the satellite bandwagon a few years ago and got good mileage out of it. However, at the end of 2011, Asianet, one of the primary channels they watch, moved from Insat 2E to Intelsat 17. The shift to the more westerly satellite had the unfortunate consequence of removing coverage from the eastern seaboard of Australia.

A lot of disappointed Malayalees in Sydney, Melbourne and Brisbane had to find alternatives for getting their Malayalam TV fix. My grandparents found their saviour in BomTV, which provided live steaming via the Internet albeit on a subscription basis. This required a Roku (or Android) device, which was provided with the initial subscription to a package. The streaming quality wasn’t as good as satellite and the device would need to occasionally buffer, but it was adequate.

Recently, my grandparents needed to replace their TV and they ended up purchasing a smart TV with Internet browsing. Also, through my grandmother’s exploratory searching, they found that they could view Malayalam (along with other Indian language) TV content online using Bharat Channels. While it wasn’t real-time TV, they could select what they wanted to watch, whenever they wanted to watch, and it was free and without ads. Poking around, I noticed the website simply pointed to YouTube for all their content. I’m unsure if it is indeed the Indian TV channels themselves posting the content (as opposed to a third-party) but either way, it’s probably a good monetisation strategy for somebody.

The Bharat Channels website is obviously designed for a desktop (and not a TV) and adds a lot of visual noise. In order to streamline their viewing experience, I created a simple website that serves up the same YouTube content, but packaged in a more user-friendly way: Malayalam TV.

Using PHP and Twitter Bootstrap (for a responsive design), the script reads a specified YouTube channel’s RSS feed and displays the content as easy to click thumbnails. The thumbnails link to a full screen version of the video. Surprisingly, an iPad’s Safari browser requires a different link: http://www.youtube.com/embed/{id} vs. http://www.youtube.com/v/{id} while an iPhone doesn’t. I also had to use CSS media queries to lock the dimensions of the thumbnails, since certain videos have quite long titles. I did look into CSS’s text-overflow: ellipsis options, but this doesn’t work across multiple lines. While there are several JavaScript truncation libraries, in trying to be as minimal as possible, I just went for the quick and crude approach. This theme also carried over into the CSS, where only used CSS from Twitter Bootstrap was included.

See the code on GitHub.

Coloured directory listings in PowerShell

By default, PowerShell outputs drab directory listings. Unfortunately, there’s no built in solution to turning on colour, but it is possible with a few steps.

  1. This StackOverflow post gets us most of the way there. Towards the bottom of the question is some code that enables colourised output. I’ve modified it to suit my purposes (see below). This code should be put in your profile (see $profile to find the location).
  2. As noted, the code depends on the New-CommandWrapper cmdlet from the Powershell Cookbook. In my PowerShell profile directory, I’ve created a Scripts directory as a sibling to the regular Modules directory and put the cmdlet there.
  3. Ensure New-CommandWrapper.ps1 is unblocked (right click, Properties, Unblock) and in your PATH. If using the Scripts directory, you can put $env:Path += ";$(Split-Path $profile)\Scripts" in your profile.
  4. Using Set-ExecutionPolicy, ensure your execution policy is set to RemoteSigned or Unrestricted.
  5. Start a new shell, type dir or ls and enjoy.

As mentioned, I’ve modified the code in the StackOverflow post to suit my needs:

  • Cyan instead of Magenta directories. Magenta hurt my eyes.
  • Ignoring colouring text files separately as yellow. I have a lot of text files using different extensions and colouring text files didn’t provide any benefit.
  • Removed Perl, Python and Ruby extensions from executables. I don’t use Perl or Python and I have more Ruby code as part of a Rails app than standalone scripts.
  • Adjusted column spacing.
  • Don’t output a length for directories. It doesn’t show in the screenshot, but it displayed a length of 1 for directories.
  • Add coloured file count along with total file size at bottom.
  • Override both dir and ls aliases.
  • Renamed functions according to PowerShell verb-noun conventions.

The code that goes in your profile is:

New-CommandWrapper Out-Default -Process {
  $regex_opts = ([System.Text.RegularExpressions.RegexOptions]::IgnoreCase)
  $compressed = New-Object System.Text.RegularExpressions.Regex(
    '\.(zip|tar|gz|rar|jar|war)$', $regex_opts)
  $executable = New-Object System.Text.RegularExpressions.Regex(
    '\.(exe|bat|cmd|msi|ps1|psm1|vbs|reg)$', $regex_opts)

  if(($_ -is [System.IO.DirectoryInfo]) -or ($_ -is [System.IO.FileInfo]))
  {
    if(-not ($notfirst))
    {
      Write-Host "`n    Directory: " -noNewLine
      Write-Host "$(pwd)`n" -foregroundcolor "Cyan"
      Write-Host "Mode        Last Write Time       Length   Name"
      Write-Host "----        ---------------       ------   ----"
      $notfirst=$true
    }

    if ($_ -is [System.IO.DirectoryInfo])
    {
      Write-Host ("{0}   {1}                {2}" -f $_.mode, ([String]::Format("{0,10} {1,8}", $_.LastWriteTime.ToString("d"), $_.LastWriteTime.ToString("t"))), $_.name) -ForegroundColor "Cyan"
    }
    else
    {
      if ($compressed.IsMatch($_.Name))
      {
        $color = "DarkGreen"
      }
      elseif ($executable.IsMatch($_.Name))
      {
        $color =  "Red"
      }
      else
      {
        $color = "White"
      }
      Write-Host ("{0}   {1}   {2,10}   {3}" -f $_.mode, ([String]::Format("{0,10} {1,8}", $_.LastWriteTime.ToString("d"), $_.LastWriteTime.ToString("t"))), $_.length, $_.name) -ForegroundColor $color
    }

    $_ = $null
  }
} -end {
  Write-Host
}

function Get-DirSize
{
  param ($dir)
  $bytes = 0
  $count = 0

  Get-Childitem $dir | Foreach-Object {
    if ($_ -is [System.IO.FileInfo])
    {
      $bytes += $_.Length
      $count++
    }
  }

  Write-Host "`n    " -NoNewline

  if ($bytes -ge 1KB -and $bytes -lt 1MB)
  {
    Write-Host ("" + [Math]::Round(($bytes / 1KB), 2) + " KB") -ForegroundColor "White" -NoNewLine
  }
  elseif ($bytes -ge 1MB -and $bytes -lt 1GB)
  {
    Write-Host ("" + [Math]::Round(($bytes / 1MB), 2) + " MB") -ForegroundColor "White" -NoNewLine
  }
  elseif ($bytes -ge 1GB)
  {
    Write-Host ("" + [Math]::Round(($bytes / 1GB), 2) + " GB") -ForegroundColor "White" -NoNewLine
  }
  else
  {
    Write-Host ("" + $bytes + " bytes") -ForegroundColor "White" -NoNewLine
  }
  Write-Host " in " -NoNewline
  Write-Host $count -ForegroundColor "White" -NoNewline
  Write-Host " files"

}

function Get-DirWithSize
{
  param ($dir)
  Get-Childitem $dir
  Get-DirSize $dir
}

Remove-Item alias:dir
Remove-Item alias:ls
Set-Alias dir Get-DirWithSize
Set-Alias ls Get-DirWithSize

Moving to a Windows JRuby on Rails development environment

tl;dr I recently set up a JRuby on Rails development environment on Windows after using Mac OS X and Ubuntu VMs. It’s noticeably faster for development and more comfortable. I’ll stick with it.

When I was first starting JRuby on Rails development, I started with a Mac OS X environment running from VirtualBox. I chose Mac OS X to emulate as close as possible what the other members in my team were using, so I could get advice on the basics without too much friction. However, Mac OS X isn’t officially supported by VirtualBox and the environment was unwieldy – no ability to resize the VM window, no shared clipboard and occasionally crashes on boot were the main drawbacks.

After a few weeks, I moved to an Ubuntu VM. This did require minor adjustment in our codebase to get it working on a development environment. VirtualBox supports Ubuntu, and the drawbacks from the Mac OS X experience were eliminated. I used this environment for a good eight months, and became quite familiar with Unix environment. The terminal took the forefront (compared to Windows), apt-get package management was fantastic and everything felt solid. However, after copying the VM to my laptop, running Rails and tests was considerably slow.

Another reason I initially avoided Windows as a development environment was the common perception at the time was that Windows was a second-class citizen in the Ruby community, and performance was lacking. Supposedly even a Ubuntu VM would outperform a native Windows environment. Having gained significant experience with JRuby on Rails over the last few months, I felt comfortable enough to try my hand at setting up a Windows development environment. Here’s what I learnt from the process:

  • Use the Java SDK instead of Java JRE to run JRuby. I would occasionally get Cannot find Java 1.5 or higher error messages when running certain gems. Ensure JAVA_HOME is set to Program Files/Java/jdk* and PATH includes Program Files/Java/jdk*/bin.
  • JRuby on Windows requires the jruby-pageant gem to interact with net/ssh.
  • We are using Eco templates and I was encountering Encoding::Compatibility Error, incompatible character encodings: UTF-8 and Windows-1252 errors when trying to compile a Javascript template that had a UTF-8 character. UTF-8 characters in ERB templates displayed correctly. I couldn’t solve this one and ended up using the escaped HTML format instead.
  • Command Prompt and Powershell don’t natively support ANSI escape characters (e.g. colour codes). So running RSpec would insert a bunch of garbage characters in the output. Turning off --color helped, but there was still some around the progress bar itself, notwithstanding the loss of coloured output. I investigated ANSICON but it seemed too complex to install and I wasn’t entirely sure I could get it working with Powershell. So I decided to install Cygwin and while doing so, thought I’d use ConEmu to unify my Cygwin and Powershell terminals under one umbrella. That’s when I found Powershell, when run through ConEmu, supports colour codes and also allowed me to navigate through irb/rails console history.
  • I found PuTTY itself more convenient than running plink through Powershell as it seems to better interact with bash. PuTTY can also be set up as a task/jump-list in ConEmu to avoid typing the full command.
  • You have to add your private RSA key to Pageant. I always wondered what the computer icon with the hat in my taskbar was for.
  • Use PowerTab with PowerShell. It uses bash-like tab completion instead of the default cycling completion.

In the end, it’s about performance and friction. In comparison with the Ubuntu VM environment, JRuby start up time is noticeably quicker although running the entire test suite is mixed – its faster when using Poltergeist/PhantomJS for integration/feature tests, while slower using Selenium (which we’ve since abandoned). This leads me to think the original performance advice of using a Unix system was based on experience with MRI rather than JRuby.

The Windows environment itself is more familiar and I can now Alt-Tab between my IDE, terminals, file manager, and browser, and move applications to different monitors. As there are no showstoppers, I’ll be sticking with this environment for the foreseeable future.

Install Oracle XE 11g R2 on Ubuntu 12.04

I’ve recently had a task of enabling Oracle database support for a JRuby on Rails application. To set up an Oracle database for use on a development environment, there were two preferred options – using Oracle DB on a virtual machine or installing Oracle XE locally.

1. Oracle DB on a virtual machine
Oracle kindly provide several VirtualBox images for download – I used the Database App Development VM, which comes with Enterprise Edition and SQL Developer, and it’s straightforward to get this running. The only catch is to enable a second network adapter (either Bridged or Host-only) in the VirtualBox settings so the VM is externally addressable. From there, keep in mind the the instance name is orcl, and sys and system passwords are oracle.

2. Installing Oracle XE locally
Oracle XE is officially supported on Windows and Red Hat Linux-based systems (the Developer Days VMs are Oracle Linux). Debian Linux-based systems don’t seem to be officially supported, but there have been efforts on installing Oracle XE on Ubuntu (my particular interest). I primarily followed Manish’s guide for the majority of the install process, but had to bring forward the step of setting environment variables. This is because Oracle XE starts immediately after the configuration step (oracle-xe configure). I also referred to another blog for guidance on setting kernel parameters. Here are my modified installation steps:

Download Oracle Database Express Edition 11g R2

Unzip archive:

unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip

Install the required packages (note I omitted unixodbc as I didn’t require it):

sudo apt-get install alien libaio1

Convert the RPM package to a DEB package:

cd Disk1
sudo alien --scripts oracle-xe-11.2.0-1.0.x86_64.rpm

Create a /sbin/chkconfig file with the following contents (requires root):

#!/bin/bash
# Oracle 11gR2 XE installer chkconfig hack for Ubuntu
file=/etc/init.d/oracle-xe
if [[ ! `tail -n1 $file | grep INIT` ]]; then
echo >> $file
echo '### BEGIN INIT INFO' >> $file
echo '# Provides: OracleXE' >> $file
echo '# Required-Start: $remote_fs $syslog' >> $file
echo '# Required-Stop: $remote_fs $syslog' >> $file
echo '# Default-Start: 2 3 4 5' >> $file
echo '# Default-Stop: 0 1 6' >> $file
echo '# Short-Description: Oracle 11g Express Edition' >> $file
echo '### END INIT INFO' >> $file
fi
update-rc.d oracle-xe defaults 80 01

Set execute privileges for the file:

sudo chmod 755 /sbin/chkconfig

Create a /etc/sysctl.d/60-oracle.conf file with the following contents (requires root):

# Oracle 11g XE kernel parameters
fs.file-max=6815744
kernel.sem=250 32000 100 128
kernel.shmmax=1073741824
net.ipv4.ip_local_port_range=9000 65000

Load the new kernel parameters:

sudo service procps start

Run the following commands:

sudo ln -s /usr/bin/awk /bin/awk
sudo mkdir /var/lock/subsys
sudo touch /var/lock/subsys/listener

To avoid MEMORY_TARGET errors:

sudo rm -rf /dev/shm
sudo mkdir /dev/shm
sudo mount -t tmpfs shmfs -o size=2048m /dev/shm

Create a /etc/rc2.d/S01shm_load file with the following contents (requires root):

#!/bin/sh
case "$1" in
start) mkdir /var/lock/subsys 2>/dev/null
	   touch /var/lock/subsys/listener
	   rm /dev/shm 2>/dev/null
	   mkdir /dev/shm 2>/dev/null
	   mount -t tmpfs shmfs -o size=2048m /dev/shm ;;
*) echo error
   exit 1 ;;
esac

Set execute privileges for the file:

sudo chmod 755 /etc/rc2.d/S01shm_load

Install the Oracle XE package:

sudo dpkg --install oracle-xe_11.2.0-2_amd64.deb

Add the following environment variables to .bashrc:

export ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe
export ORACLE_SID=XE
export NLS_LANG=`$ORACLE_HOME/bin/nls_lang.sh`
export ORACLE_BASE=/u01/app/oracle
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
export PATH=$ORACLE_HOME/bin:$PATH

Reload the bash profile:

source ~/.bashrc

Run the Oracle XE configuration:

sudo /etc/init.d/oracle-xe configure

Oracle XE should now be installed.

If anything goes wrong during installation, to uninstall:

sudo -s
/etc/init.d/oracle-xe stop
ps -ef | grep oracle | grep -v grep | awk '{print $2}' | xargs kill
dpkg --purge oracle-xe
rm -r /u01
rm /etc/default/oracle-xe
update-rc.d -f oracle-xe remove

I’ve used these instructions on two Ubuntu environments (my own VM and an Amazon EC2 instance) and Oracle XE is running happily.

History of the Bible

I ran a different type of study with our church youth group today where we investigated the history of the Bible. It was a combination of a trivia-style and lecture-style presentation and I’ve uploaded the notes I used. It took a little while to put it together – most of the time was spent reading on the Internet and distilling the important (and fun) facts – history can be a little dry. I have included references for most of the resources I used except Wikipedia (which is pretty much the foundation for most of the material). Having done this, I do have a greater appreciation for biblical scholars and academics.