codesnip

Extra tricks for linux on your arcade machine

Want your linux arcade machine to boot right into, and shut down from your attract-mode front end?

Change to OpenBox as your window manager:

log out, select openbox above
log in
reboot

Add attract mode to autostart with openbox

vim ~/.config/openbox/autostart

paste this:

#AutoStart Attract-Mode
attract

:wq to save and exit vim

Hide the UI even further by changing the default colors to black

sudo vim /usr/lib/x86_64-linux-gnu/openbox-autostart

change the line under
# Set a background color
from
BG=””
to
BG=”#000000″

and this line:
#test -z $BG || $BG -solid “#303030”
to
#test -z $BG || $BG -solid “#000000”

Enable shutdown directly from exiting attract-mode:

sudo visudo -f /etc/sudoers

NOTE: this has implications, it basically allows everyone to do super user commands without a password. In the case of my arcade cab, I don’t care. Be aware of the risks.


#!! IMPORTANT: !! – PUT THIS AT THE END OF THE FILE AFTER THE LAST LINE OR YOU WILL RISK BREAKING YOUR ACCESS.

username ALL=(ALL) NOPASSWD: ALL

Then in attract mode:

Set: exit command

sudo shutdown now

img_583477c870c7e

How do I install Lubuntu 16 for MAME Cabinet?

Lubuntu 16.04, MAME 0.179, and Attract-mode 2.20 on older hardware:

We’re still using the old HP DV9700 series entertainment laptop for this example, it’s pretty old at this stage..

Here are some details:

ManufacturerHewlett Packard
ModelDV9700 
Processor
Intel(R) Core(TM)2 Duo CPU T5550 @ 1.83GHz
Memory3GB DDR2 SDRAM
Hard Drive250GB 7200RPM (Replaced with new)
MediaBluRay / DVD-RW+-
Video Memory1024MB GDDR2
Video GPUNVIDIA Corporation G86M [GeForce 8600M GS]

I just pulled the hard drive so the hyperspin install could stay intact.


Step 1: Make your linux installer.

You’ll need an empty flash drive of at least 4G capacity, I used unetbootin and created a lubuntu installer. Here’s how:

Get the UNetbootin package for your operating system: https://unetbootin.github.io/

Start UNetbootin (with administrator level permissions if needed).

Select the Lubuntu Distrubution 16.04_live (Select _x64 if you want to install 64 bit mame, which we do in this case.)

Be sure to select the proper USB drive mountpoint above.

If your usb flash disk has been used before, it’s best to format it first.

If you have to format flash media I recommend the official SD formatter at sdcard.org

UNetbootin will load the bootable lubuntu image onto that USB stick and tell you when it’s done.


Step 2: Boot the linux installer and install Lubuntu:

Most semi-ancient machines can boot to USB volumes, but some need a special key or bios setting enabled.
On this machine it was pressing f10 at boot time to select a boot device.

I’ll link outward to the official docs for installing lubuntu, they did a great job at explaining it:

note: it’s easiest if you DON’T encrypt homedir, and you tell it to auto-login your user.

https://help.ubuntu.com/community/Lubuntu/InstallingLubuntu

The time it takes depends a lot on your hardware and the speed of your flash drive. But it’s not too bad to perform the install.


Step 3: Boot up and install some tweaks and tools: (Technically Optional)

Some of these are just things I did that I find useful for managing the system.

  • remote SSH access for remote management
  • VNC for remote-screen access over the network
  • SSH Greeter (stolen from RetroPie)

Install OpenSSH server for remote management:

Open LXterminal and use these commands to install and configure.

sudo apt update
sudo apt install openssh-server
chkconfig sshd on
ufw allow 22

the “ufw” command allows connections to port 22 through the firewall

You should be able to connect to your lubuntu host from the network, use puTTY on windows or terminal on mac or linux.

A quick way to find your ip address on lubuntu is:

 ifconfig |grep -w inet

Then, you can use your remote machine to connect over SSH

ssh user@YOUR_LUBUNTU_HOST

x11VNC to remotely manage system:

Open LXterminal and use these commands to install and configure.

sudo apt install x11vnc

This is probably a machine on your home network, but you should create a password for your user:

x11vnc -usepw

This command will set the password file on it’s first run.

Your user should already be set to auto-login, so you should be able to start x11VNC remotely when you need it.
SSH to your machine, Start X11 VNC with Password, use VNC client to log into it.

The following command will start an x11vnc session in your ssh session or terminal.

x11vnc -usepw

by default it should open port 5900 for VNC


SSH Greeter (Super Optional)

This is completely unnecessary. But I like to see this info when I first log into SSH. When you log into a RetroPie system with the SSH Greeter tweak installed you get this nice ascii art greeting with some quick system facts. I’ve modified it for this install.

At a glance I can see if I have updates, What my filesystem space looks like, How long the machine has been up, Temps etc.

There’s a hidden file in your Lubuntu user’s home directory that you’ll want to paste the code below into.

NOTE: you can seriously screw up your install if you do this wrong, if you’re not comfortable with linux, you might want to skip this step.

Do this from an SSH session or Terminal window:
I prefer editing text in VIM so I installed it first (sudo apt-get install vim) but you can use Nano or some other text editor if you prefer.

cp ~/.bashrc ~/.bashrc.backup
sudo vim ~/.bashrc

Next, scroll to the absolute bottom of the file. You’ll want to paste the following code in (Press “I” to enter VIM insert mode first):

# QUASICADE PROFILE START

function quasicade_welcome() {
    local upSeconds="$(/usr/bin/cut -d. -f1 /proc/uptime)"
    local secs=$((upSeconds%60))
    local mins=$((upSeconds/60%60))
    local hours=$((upSeconds/3600%24))
    local days=$((upSeconds/86400))
    local UPTIME=$(printf "%d days, %02dh%02dm%02ds" "$days" "$hours" "$mins" "$secs")

    # calculate rough CPU and GPU temperatures:
    local cpuTempC
    local cpuTempF
    local gpuTempC
    local gpuTempF
    if [[ -f "/sys/class/thermal/thermal_zone0/temp" ]]; then
        cpuTempC=$(($(cat /sys/class/thermal/thermal_zone0/temp)/1000)) && cpuTempF=$((cpuTempC*9/5+32))
    fi

    if [[ -f "/opt/vc/bin/vcgencmd" ]]; then
        if gpuTempC=$(/opt/vc/bin/vcgencmd measure_temp); then
            gpuTempC=${gpuTempC:5:2}
            gpuTempF=$((gpuTempC*9/5+32))
        else
            gpuTempC=""
        fi
    fi

    local df_out=()
    local line
    while read line; do
        df_out+=("$line")
    done < <(df -h /) local rst="$(tput sgr0)" local fgblk="${rst}$(tput setaf 0)" # Black - Regular local fgred="${rst}$(tput setaf 1)" # Red local fggrn="${rst}$(tput setaf 2)" # Green local fgylw="${rst}$(tput setaf 3)" # Yellow local fgblu="${rst}$(tput setaf 4)" # Blue local fgpur="${rst}$(tput setaf 5)" # Purple local fgcyn="${rst}$(tput setaf 6)" # Cyan local fgwht="${rst}$(tput setaf 7)" # White local bld="$(tput bold)" local bfgblk="${bld}$(tput setaf 0)" local bfgred="${bld}$(tput setaf 1)" local bfggrn="${bld}$(tput setaf 2)" local bfgylw="${bld}$(tput setaf 3)" local bfgblu="${bld}$(tput setaf 4)" local bfgpur="${bld}$(tput setaf 5)" local bfgcyn="${bld}$(tput setaf 6)" local bfgwht="${bld}$(tput setaf 7)" local logo=( "${fggrn} _ ${fgrst} " "${fggrn} /\ \ ${fgrst} " "${fggrn} / \ \ ${fgrst} " "${fggrn} / /\ \ \ ${fgrst} " "${fggrn} / / /\ \ \ ${fgrst} " "${fggrn} / / / \ \_\ ${fgrst}" "${fggrn} / / / _ / / / ${fgrst}" "${fggrn} / / / /\ \/ / ${fgrst} " "${fggrn} / / /__\ \ \/ ${fgrst} " "${fggrn}/ / /____\ \ \ ${fgrst} " "${fggrn}\/________\_\/ ${fgrst} " "${fggrn} ${fgrst}" ) local out local i for i in "${!logo[@]}"; do out+=" ${logo[$i]} " case "$i" in 0) out+="${fggrn}$(date +"%A, %e %B %Y, %r")" ;; 1) out+="${fggrn}$(uname -srmo)" ;; 3) out+="${fgylw}${df_out[0]}" ;; 4) out+="${fgwht}${df_out[1]}" ;; 5) out+="${fgred}Uptime.............: ${UPTIME}" ;; 6) out+="${fgred}Memory.............: $(grep MemFree /proc/meminfo | awk {'print $2'})kB (Free) / $(grep MemTotal /proc/meminfo | awk {'print $2'})kB (Total)" ;; 7) out+="${fgred}Running Processes..: $(ps ax | wc -l | tr -d " ")" ;; 8) out+="${fgred}IP Address.........: $(ip route get 8.8.8.8 2>/dev/null | head -1 | cut -d' ' -f8)"
                ;;
            9)
                out+="Temperature........: CPU: $cpuTempC°C/$cpuTempF°F GPU: $gpuTempC°C/$gpuTempF°F"
                ;;
            10)
                out+="${fgwht}I am quasicade, Feed me quarters.${fgrst}"
                ;;
        esac
        out+="\n"
    done
    echo -e "\n$out"
}

quasicade_welcome
# QUASICADE PROFILE END

Press ESC to exit INSERT mode when your modifications are done.

Next type “:wq” to write the file and quit VIM
You should see the greeter on the next login.

If you can’t see the CPU temp (and want to) you need to install the ‘lm-sensors’ modules

sudo apt-get install lm-sensors
sudo sensors-detect

NOTE: if something goes sideways with your bashrc file, restore the backup we made above like this:

cp ~/.bashrc.backup ~/.bashrc

Step 4: Install MAME and Attract-Mode

First we need to install the PPA repository for MAME, then update our repo list, then install MAME:

sudo add-apt-repository ppa:c.falco/mame && sudo apt-get update && sudo apt-get install mame

Then, we’ll do the same for Attract-Mode:

sudo add-apt-repository ppa:daveg/attract &&; sudo apt-get update && sudo apt-get install attract

Configure

This could be 30 posts in itself, and really depends on your particular goals.

I will link the readme docs and wikis for Attract-mode and MAME here:

http://attractmode.org/docs/Readme.html

http://docs.mamedev.org/

I will cover my experience, tips, and findings on configuring and tweaking the various aspects of MAME and Front-end on linux in future posts.

You’re probably best to start by thinking about what you want your machine to be.

I’ve decided to refine the quasicade into a gallery of my most favorite machines. While they’re all there, I’ll only be presenting 80 or so in the front-end.

Good luck!

modern-cab-horizontal-wip-v8

Vade Retro Microsoft. Hello Linux.

It’s been some time since an update on the Quasicade.

It’s been sitting in the garage lonely. We’ve got family coming for the holidays so I figured I’d bring it out and get it running. Then, I remembered the pain points. It ran windows under all that hyperspin. I’d always lamented that I couldn’t get a flashy front-end on linux.

Enter Attract-mode.

attractmodelogo

It’s not as feature heavy or as community heavy as Hyperspin. But, it’s not as, heavy.

The layouts are written in Squirrel, it’s pretty hyperspin-theme compatible. All around, it’s great.

This finally allowed me to feel confident I could get a nicely designed front end on linux. So I ripped out the hyperspin drive from the quasicade and tossed it somewhere safe, just in case I wasn’t able to get the open source machine tuned in time. I at least knew I could get the clunky hyperspin install back up again.

Choosing a distro

I wanted something light, without extra fluff if possible, and something that was pretty supported. I didn’t have time or desire to screw around building my own graphics drivers.

I tested #! linux (crunchbang) on my old EEEPC Nintendo emulator, I got RetroPi working on it, but I felt I needed to go 64Bit and something that was still alive and supported.
lubuntu
I chose Lubuntu: Lubuntu is a fast and lightweight operating system. The core of the system is based on Linux and Ubuntu. Lubuntu uses the minimal desktop LXDE, and a selection of light applications. We focus on speed and energy-efficiency. Because of this, Lubuntu has very low hardware requirements.

 

So, How’d it go?

Great, actually. Over the next few posts, I’ll be going into detail on what I did to build and configure MAME 0.179, Attract-Mode, and Lubuntu. Some tricks I learned from the project that you might want in your build, and a few things I stole from the RetroPie project to use on this machine.

codesnip

Migrate shared hosting email to a new host with imapsync

One of the major pains of shared hosting migration is user email on the system. This process enables a fairly easy method of migration using an application on the command line of a CentOS server (or any other linux host) as an intermediary.

[hr top=”0″ bottom=”18″ /]

Log into the server you wish to be the migrator:

ssh user@YOURLINUXBOX.FQDN.COM

Next we must install imapsync, the next two commands are dependent on your architecture, choose one:

RHEL based distro:

sudo yum -y install imapsync

Debian based distro:

sudo apt-get -y install imapsync

It will install the perl dependencies and the imapsync CLI application.

[hr top=”0″ bottom=”18″ /]

Next we’ll issue the sync commands.

Here are some of the options you can use (from the manpage):

[hr top=”0″ bottom=”18″ /]

usage: /usr/bin/imapsync [options]

Several options are mandatory. 

--dry                  : Makes imapsync doing nothing, just print what would 
                         be done without --dry.

--host1        : Source or "from" imap server. Mandatory.
--port1           : Port to connect on host1. Default is 143, 993 if --ssl1
--user1        : User to login on host1. Mandatory.
--showpasswords        : Shows passwords on output instead of "MASKED".
                         Useful to restart a complete run by just reading the log.
--password1    : Password for the user1.
--host2        : "destination" imap server. Mandatory.
--port2           : Port to connect on host2. Default is 143, 993 if --ssl2
--user2        : User to login on host2. Mandatory.
--password2    : Password for the user2.

--passfile1    : Password file for the user1. It must contain the 
                         password on the first line. This option avoids to show
                         the password on the command line like --password1 does.
--passfile2    : Password file for the user2. Contains the password.

--ssl1                 : Use a SSL connection on host1.
--ssl2                 : Use a SSL connection on host2.
--tls1                 : Use a TLS connection on host1.
--tls2                 : Use a TLS connection on host2.
--timeout         : Connections timeout in seconds. Default is 120.
                         0 means no timeout.

--authmech1    : Auth mechanism to use with host1:
                         PLAIN, LOGIN, CRAM-MD5 etc. Use UPPERCASE.
--authmech2    : Auth mechanism to use with host2. See --authmech1

--authuser1    : User to auth with on host1 (admin user). 
                         Avoid using --authmech1 SOMETHING with --authuser1.
--authuser2    : User to auth with on host2 (admin user).
--proxyauth1           : Use proxyauth on host1. Requires --authuser1.
                         Required by Sun/iPlanet/Netscape IMAP servers to
                         be able to use an administrative user.
--proxyauth2           : Use proxyauth on host2. Requires --authuser2.

--authmd51             : Use MD5 authentification for host1.
--authmd52             : Use MD5 authentification for host2.
--domain1      : Domain on host1 (NTLM authentication).
--domain2      : Domain on host2 (NTLM authentication).


--folder       : Sync this folder.
--folder       : and this one, etc.
--folderrec    : Sync this folder recursively.
--folderrec    : and this one, etc.

--folderfirst  : Sync this folder first. --folderfirst "Work"
--folderfirst  : then this one, etc.
--folderlast   : Sync this folder last. --folderlast "[Gmail]/All Mail"
--folderlast   : then this one, etc.

--nomixfolders         : Do not merge folders when host1 is case sensitive
                         while host2 is not (like Exchange). Only the first
                         similar folder is synced (ex: Sent SENT sent -> Sent).

--skipemptyfolders     : Empty host1 folders are not created on host2.

--include       : Sync folders matching this regular expression
--include       : or this one, etc.
                         in case both --include --exclude options are
                         use, include is done before.
--exclude       : Skips folders matching this regular expression
                         Several folders to avoid:
			  --exclude 'fold1|fold2|f3' skips fold1, fold2 and f3.
--exclude       : or this one, etc.

--regextrans2   : Apply the whole regex to each destination folders.
--regextrans2   : and this one. etc.
                         When you play with the --regextrans2 option, first
                         add also the safe options --dry --justfolders
                         Then, when happy, remove --dry, remove --justfolders.
                         Have in mind that --regextrans2 is applied after prefix 
                         and separator inversion.

--tmpdir       : Where to store temporary files and subdirectories.
                         Will be created if it doesn't exist.
			 Default is system specific, Unix is /tmp but
                         it's often small and deleted at reboot.
                         --tmpdir /var/tmp should be better.
--pidfile      : The file where imapsync pid is written.
--pidfilelocking       : Abort if pidfile already exists. Usefull to avoid 
                         concurrent transfers on the same mailbox.

--nolog                : Turn off logging on file
--logfile      : Change the default logfile pathname and filename.

--prefix1      : Remove prefix to all destination folders 
                         (usually INBOX. or INBOX/ or an empty string "")
                         you have to use --prefix1 if host1 imap server
                         does not have NAMESPACE capability, all other
                         cases are bad.
--prefix2      : Add prefix to all host2 folders. See --prefix1
--sep1         : Host1 separator in case NAMESPACE is not supported.
--sep2         : Host2 separator in case NAMESPACE is not supported.

--skipmess      : Skips messages maching the regex.
                         Example: 'm/[\x80-ff]/' # to avoid 8bits messages.
                         --skipmess is applied before --regexmess
--skipmess      : or this one, etc.

--disarmreadreceipts   : Disarms read receipts (host2 Exchange issue)

--regexmess     : Apply the whole regex to each message before transfer.
                         Example: 's/\000/ /g' # to replace null by space.
--regexmess     : and this one, etc.

--regexflag     : Apply the whole regex to each flags list.
                         Example: 's/"Junk"//g' # to remove "Junk" flag.
--regexflag     : and this one, etc.

--delete               : Deletes messages on host1 server after a successful 
                         transfer. Option --delete has the following behavior: 
                         it marks messages as deleted with the IMAP flag 
                         \Deleted, then messages are really deleted with an 
                         EXPUNGE IMAP command.

--delete2              : Delete messages in host2 that are not in
                         host1 server. Useful for backup or pre-sync.
--delete2duplicates    : Delete messages in host2 that are duplicates.
                         Works only without --useuid since duplicates are 
                         detected with an header part of each message.

--delete2folders       : Delete folders in host2 that are not in host1 server. 
                         For safety, first try it like this (it is safe):
			 --delete2folders --dry --justfolders --nofoldersizes
--delete2foldersonly   : Deleted only folders matching regex.
                         Example: --delete2foldersonly "/^Junk$|^INBOX.Junk$/"
--delete2foldersbutnot : Do not delete folders matching regex.
                         Example: --delete2foldersbutnot "/Tasks$|Contacts$|Foo$/"
--noexpunge            : Do not expunge messages on host1.
                         Expunge really deletes messages marked deleted.
                         Expunge is made at the beginning, on host1 only. 
                         Newly transferred messages are also expunged if 
			 option --delete is given.
                         No expunge is done on host2 account (unless --expunge2)
--expunge1             : Expunge messages on host1 after messages transfer.
--expunge2             : Expunge messages on host2 after messages transfer.
--uidexpunge2          : uidexpunge messages on the host2 account
                         that are not on the host1 account, requires --delete2
--nomixfolders         : Avoid merging folders that are considered different on
                         host1 but the same on destination host2 because of 
                         case sensitivities and insensitivities.

--syncinternaldates    : Sets the internal dates on host2 same as host1.
                         Turned on by default. Internal date is the date
			 a message arrived on a host (mtime).
--idatefromheader      : Sets the internal dates on host2 same as the 
                         "Date:" headers.

--maxsize         : Skip messages larger  (or equal) than  bytes
--minsize         : Skip messages smaller (or equal) than  bytes
--maxage          : Skip messages older than  days.
                         final stats (skipped) don't count older messages
			 see also --minage
--minage          : Skip messages newer than  days.
                         final stats (skipped) don't count newer messages
                         You can do (+ are the messages selected):
                         past|----maxage+++++++++++++++>now
                         past|+++++++++++++++minage---->now
                         past|----maxage+++++minage---->now (intersection)
                         past|++++minage-----maxage++++>now (union)

--search       : Selects only messages returned by this IMAP SEARCH 
                         command. Applied on both sides.
--search1      : Same as --search for selecting host1 messages only.
--search2      : Same as --search for selecting host2 messages only.
                         --search CRIT equals --search1 CRIT --search2 CRIT

--exitwhenover    : Stop syncing when total bytes transferred reached.
                         Gmail per day allows 2500000000 down 500000000 upload.

--maxlinelength   : skip messages with a line length longer than  bytes.
                         RFC 2822 says it must be no more than 1000 bytes.

--useheader    : Use this header to compare messages on both sides.
                         Ex: Message-ID or Subject or Date.
--useheader      and this one, etc.

--subscribed           : Transfers subscribed folders.
--subscribe            : Subscribe to the folders transferred on the 
                         host2 that are subscribed on host1. On by default.
--subscribeall         : Subscribe to the folders transferred on the 
                         host2 even if they are not subscribed on host1.

--nofoldersizes        : Do not calculate the size of each folder in bytes
                         and message counts. Default is to calculate them.
--nofoldersizesatend   : Do not calculate the size of each folder in bytes
                         and message counts at the end. Default is on.
--justfoldersizes      : Exit after having printed the folder sizes.

--syncacls             : Synchronises acls (Access Control Lists).
--nosyncacls           : Does not synchronize acls. This is the default.
                         Acls in IMAP are not standardized, be careful.

--usecache             : Use cache to speedup.
--nousecache           : Do not use cache. Caveat: --useuid --nousecache creates
                         duplicates on multiple runs.
--useuid               : Use uid instead of header as a criterium to recognize 
                         messages. Option --usecache is then implied unless 
                         --nousecache is used.  

--debug                : Debug mode.
--debugcontent         : Debug content of the messages transfered.
--debugflags           : Debug flags.
--debugimap1           : IMAP debug mode for host1. imap debug is very verbose.
--debugimap2           : IMAP debug mode for host2.
--debugimap            : IMAP debug mode for host1 and host2.

--tests                : Run non-regression tests.
--testslive            : Run a live test with test1.lamiral.info imap server. 
                         Useful to check the basics. Needs internet connexion.

--version              : Print software version.
--noreleasecheck       : Do not check for new imapsync release (a http request).
--releasecheck         : Check for new imapsync release (a http request).
--justconnect          : Just connect to both servers and print useful
                         information. Need only --host1 and --host2 options.
--justlogin            : Just login to both host1 and host2 with users 
                         credentials, then exit.
--justfolders          : Do only things about folders (ignore messages).

--help                 : print this help.

Example: to synchronize imap account "test1" on "test1.lamiral.info"
                    to  imap account "test2" on "test2.lamiral.info"
                    with test1 password "secret1"
                    and  test2 password "secret2"

/usr/bin/imapsync \
   --host1 test1.lamiral.info --user1 test1 --password1 secret1 \
   --host2 test2.lamiral.info --user2 test2 --password2 secret2

[hr top=”0″ bottom=”18″ /]

OK, thats a lot of options. For this situation let’s assume we’re migrating from one shared hosting to another (e.g. bluehost->inmotionhosting)

You should create the account you wish to sync on the target host. Use your cpanel to create the same user and password at the target.
In this example that’s USERNAME@DOMAIN.COM with the password ‘PASSWORD’.

Find out what your shared host uses for IMAP connections. Also make note whether they offer Secure IMAP (over SSL or TLS) you’ll need that info for the next command.

Once you’ve created the target account and imapsync is installed on the intermediary server you can dry-run sync the imap trees with one command:

imapsync --dry --ssl2 --host1 YOURIMAP.bluehost.com --user1 USERNAME@DOMAIN.com --password1 'PASSWORD' --host2 YOURSECUREIMAP.inmotionhosting.com --user2 USERNAME@DOMAIN.com --password2 'PASSWORD'

In order, here’s the breakdown of the command:

[hr top=”0″ bottom=”18″ /]

–dry creates a dry run, this performs a non-destructive test, if you see the output you’re looking for you can continue. Otherwise it protects you from a bad sync.

–ssl2 says that the new host (host2) is using ssl, you can specify the same flag for host1 by adding –ssl1

–host1 YOURIMAP.bluehost.com specifies the name of the host that holds the mailbox to be migrated. Verify whether or not you need ssl/tls for this connection.

–user1 USERNAME@DOMAIN.com this will be the username to log in to host1’s email box

–password1 ‘PASSWORD’  you should add the single quotes especially if there are special characters in the password.

–host2 YOURSECUREIMAP.inmotionhosting.com sets the second host, or the target of the migration. also, verify ssl/tls and specify in the initial flags if needed.

–user2 USERNAME@DOMAIN.com this in most cases will be the exact same username and password, but created on the new host (host2)

–password2 ‘PASSWORD’ the password for the target account on host2

Update that command string with the required flags and info, and run it. You should see output like this:

[hr top=”0″ bottom=”18″ /]

imapsync --dry --ssl2 --host1 YOURIMAP.bluehost.com --user1 USER@DOMAIN.com --password1 'PASSWORD' --host2 YOURSECUREIMAP.inmotionhosting.com --user2 USER@DOMAIN.com --password2 'PASSWORD'
Transfer started at Wed Nov  4 08:27:10 2015
PID is 24597
Log file is LOG_imapsync/2015_11_04_08_27_10_USER@DOMAIN.com.txt  ( to change it, use --logfile filepath ; or use --nolog to turn off logging )
$RCSfile: imapsync,v $ $Revision: 1.637 $ $Date: 2015/04/01 01:36:37 $ 
Here is a [linux] system (Linux YOURLINUXBOX.FQDN.com 2.6.32-573.7.1.el6.x86_64 #1 SMP Tue Sep 22 22:00:00 UTC 2015 x86_64)
With perl 5.10.1 Mail::IMAPClient  3.34
Command line used:
/usr/bin/imapsync --dry --ssl2 --host1 YOURIMAP.bluehost.com --user1 USER@DOMAIN.com --password1 MASKED --host2 YOURSECUREIMAP.inmotionhosting.com --user2 USER@DOMAIN.com --password2 MASKED
Temp directory is /tmp  ( to change it use --tmpdir dirpath )
PID file is /tmp/imapsync.pid ( to change it use --pidfile filepath ; to avoid it use --pidfile "" )
Modules version list:
Mail::IMAPClient     3.34
IO::Socket           1.31
IO::Socket::IP       ?
IO::Socket::INET     1.31
IO::Socket::SSL      1.31
Net::SSLeay          1.35
Compress::Zlib       2.021
Digest::MD5          2.39
Digest::HMAC_MD5     1.01
Digest::HMAC_SHA1    1.01
Term::ReadKey        2.30
File::Spec           3.3
Time::HiRes          1.9721
Unicode::String      2.09
IO::Tee              0.64
File::Copy::Recursive 0.38
Authen::NTLM         1.09
URI::Escape          3.29
Data::Uniqid         0.12
( use --no-modules_version to turn off printing this Perl modules list )
Info: turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
Info: will try to use LOGIN authentication on host1
Info: will try to use LOGIN authentication on host2
Info: imap connexions timeout is 120 seconds
Host1: IMAP server [YOURIMAP.bluehost.com] port [143] user [USER@DOMAIN.com]
Host2: IMAP server [YOURSECUREIMAP.inmotionhosting.com] port [993] user [USER@DOMAIN.com]
Host1: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
Host1: YOURIMAP.bluehost.com says it has CAPABILITY for AUTHENTICATE LOGIN
Host1: success login on [YOURIMAP.bluehost.com] with user [USER@DOMAIN.com] auth [LOGIN]
Host2: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc.  See COPYING for distribution information.
Host2: YOURSECUREIMAP.inmotionhosting.com says it has NO CAPABILITY for AUTHENTICATE LOGIN
Host2: success login on [YOURSECUREIMAP.inmotionhosting.com] with user [USER@DOMAIN.com] auth [LOGIN]
Host1: state Authenticated
Host2: state Authenticated
Host1 capability: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA
Host2 capability: IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN ACL ACL2=UNION
Host1: separator given by NAMESPACE: [.]
Host2: separator given by NAMESPACE: [.]
Host1: prefix given by NAMESPACE: [INBOX.]
Host2: prefix given by NAMESPACE: [INBOX.]
Host1 separator and prefix: [.][INBOX.]
Host2 separator and prefix: [.][INBOX.]

++++ Listing folders
All foldernames are presented between brackets like [X] where X is the foldername.
When a foldername contains non-ASCII characters it is presented in the form
[X] = [Y] where
X is the imap foldername you have to use in command line options and
Y is the uft8 output just printed for convenience, to recognize it.

Host1 folders list:
[INBOX]
[INBOX.Drafts]
[INBOX.Junk]
[INBOX.Sent]
[INBOX.Trash]

Host2 folders list:
[INBOX]
[INBOX.Drafts]
[INBOX.Junk]
[INBOX.Sent]
[INBOX.Trash]


Folders sizes before the synchronization.
You can remove foldersizes listings by using "--nofoldersizes" and  "--nofoldersizesatend"
but then you will also loose the ETA (Estimation Time of Arrival) given after each message copy.
++++ Calculating sizes on Host1
Host1 folder [INBOX]                             Size: 538025884 Messages:  8023 Biggest:  30508221
Host1 folder [INBOX.Drafts]                      Size:         0 Messages:     0 Biggest:         0
Host1 folder [INBOX.Junk]                        Size:      3748 Messages:     1 Biggest:      3748
Host1 folder [INBOX.Sent]                        Size: 108469842 Messages:   170 Biggest:  30507437
Host1 folder [INBOX.Trash]                       Size:     87568 Messages:     8 Biggest:     30073
Host1 Nb messages:            8202 messages
Host1 Total size:        646587042 bytes (616.633 MiB)
Host1 Biggest message:    30508221 bytes (29.095 MiB)
Host1 Time spent:              4.3 seconds
++++ Calculating sizes on Host2
Host2 folder [INBOX]                             Size:         0 Messages:     0 Biggest:         0
Host2 folder [INBOX.Drafts]                      Size:         0 Messages:     0 Biggest:         0
Host2 folder [INBOX.Junk]                        Size:         0 Messages:     0 Biggest:         0
Host2 folder [INBOX.Sent]                        Size:         0 Messages:     0 Biggest:         0
Host2 folder [INBOX.Trash]                       Size:         0 Messages:     0 Biggest:         0
Host2 Nb messages:               0 messages
Host2 Total size:                0 bytes (0.000 KiB)
Host2 Biggest message:           0 bytes (0.000 KiB)
Host2 Time spent:              0.1 seconds
++++ Looping on each folder
[INBOX]                             -> [INBOX]                            
Subscribing to folder INBOX on destination server
msg INBOX/1 copying to INBOX 	(not really since --dry mode)
msg INBOX/2 copying to INBOX 	(not really since --dry mode)
msg INBOX/3 copying to INBOX 	(not really since --dry mode)
msg INBOX/4 copying to INBOX 	(not really since --dry mode)

### TRUNCATED HERE ###

Towards the end there, you see it starts copying messages, but it’s simulated because of dry run.

If you feel the output matches your desired outcome, you can stop the process with CTRL+C and remove –dry from the command to begin sync.

Once the sync has completed, verify the new account by logging into webmail. If everything looks good, you can change your DNS settings to point to the new server.

Because of propagation, be sure to monitor the old account for a few days before deleting it, some mail services may cache the DNS lookup and deliver to the old box. This is unfortunately unavoidable.

codesnip

Adding a new FileDaemon to Bacula

Adding a new FileDaemon to backup a new host (not already serving files to Bacula) is relatively similar between Windows and Unix Compatible.

You must first install the FD Client on the server you wish to back up.

This guide covers windows hosts, but linux hosts are primarily the same setup.

[hr top=”0″ bottom=”18″ /]

Install the FD Service client on the server.

Download the appropriate version of the bacula-FD (Currently running Bacula 7.0.5 with windows FD of 5.2.10)

Windows FD 5.2.10 download

Run the installer as an administrator in windows. Once the installer completes, you must locate and edit the configuration files:

You can usually find these in windows by clicking Start -> All Programs -> Bacula -> Configuration -> Edit *SOMETHING* Configuration.

We’ll stick to Client Configuration for this guide.

 

It opens in a text editor. Here are the relevant sections to edit:

#
# "Global" File daemon configuration specifications
#
FileDaemon {                            # this is me
  Name = SERVERNAME-fd
  FDport = 9102                # where we listen for the director
  WorkingDirectory = "C:\\Program Files\\Bacula\\working"
  Pid Directory = "C:\\Program Files\\Bacula\\working"
# Plugin Directory = "C:\\Program Files\\Bacula\\plugins"
  Maximum Concurrent Jobs = 10
}

Under the FileDaemon section, edit your global name field to match whatever is expected in the Director configuration.

[hr top=”0″ bottom=”18″ /]

Next we need to set the director name and password:

#
# List Directors who are permitted to contact this File daemon
#
Director {
  Name = bacula-dir
  Password = "XxXXxxXXxxxxxXXXxxxxXX"
}

Note, the name of bacula-dir is the default. Make sure it matches the main server director’s name.
The password field can be unique to this FD client, but must match in the main bacula director configuration file.

Save these files to the local machine, restart the bacula FD service.

[hr top=”0″ bottom=”18″ /]

Add the new FileDaemon to Bacula Director

ssh root@BACULA.YOURDOMAIN.com

Then, super-user edit the bacula-dir.conf file:

sudo nano /etc/bacula/bacula-dir.conf

Next we’ll locate the client config section and edit as follows:

Find a record to duplicate and copy/paste it to make your modifications.

# SERVERNAME Server
Client {
  Name = SERVERNAME-fd
  Address = SERVERNAME.YOURDOMAIN.com
  FDPort = 9102
  Catalog = MyCatalog
  Password = "XxXXxxXXxxxxxXXXxxxxXX"	 # password for FileDaemon
  File Retention = 7 days            # one week
  Job Retention = 7 days           # one week
  AutoPrune = no                    # Prune expired Jobs/Files
}

Edit the comment to match your server name.
Edit the Name field to match your FD’s name.
Edit the address to the IP or FQDN of the server you wish to add.
Verify that the password matches exactly.

Save changes to bacula-dir.conf and restart the director:

sudo service bacula-dir restart

You can now proceed to follow the steps for adding a new backup location to your rotation.

codesnip

Adding a new backup source to Bacula

To add a new backup source to Bacula, use the following procedure.

Log in to your bacula installation:

#ssh root@bacula.YOURFQDN.com

On bacula.YOURFQDN.com:
(ALWAYS back up your working config file first!)

#sudo cp /etc/bacula/bacula-dir.conf /etc/bacula/bacula-dir.conf.bak

Edit /etc/bacula/bacula-dir.conf

#sudo nano /etc/bacula/bacula-dir.conf

There are a few notable sections we must edit to add a new target:

  • Job definition
  • File Set
  • Client File Services (if not already added to the director)

[hr top=”0″ bottom=”18″ /]

The Job definition record is typically as follows, you can usually duplicate and modify:

#Backup SOMETHING on SERVERNAME
Job {
Name = "Backup SOMETHING share"
JobDefs = DefaultJob
Type = Backup
Level = Incremental
Client = servername-fd
FileSet = "SERVERNAME-SOMETHING"
Schedule = "WeeklyCycle"
Storage = Storage01
Pool = BaculaPool
}
  • Replace SOMETHING with a description of what you’re backing up.
    1. That could be the name of a share (e.g. Clients).
  • Adjust the Name field for the friendly name of the backup job.
  • Edit the Client field to match the exact name of the bacula FileDaemon for a given server (See Adding a new FileDaemon).
  • Edit the FileSet name to exactly match the name specified in the FileSet directive later in the configuration file.
  • Edit the Storage field according to which storage pool you’d like. Be careful not to over-fill a pool with backups
    1. Options in this example:
      1. Storage01 = /iscsi/storage01/*
      2. Storage02 = /iscsi/storage02/*
      3. Storage03 = /iscsi/storage03/*
  • Match the Pool reference to the appropriate storage recycle pool to the storage you chose:
    1. Storage01 = BaculaPool
    2. Storage02 = BaculaPool2
    3. Storage03 = BaculaPool3

[hr top=”0″ bottom=”18″ /]

Locate the file set closest to the config you duplicated for backup job. Please note, these are different depending on backup of Linux/unix or Windows based file formats:

Windows FileSet:

#SERVERNAME SOMETHING Backup Set
FileSet {
 Name = "SERVERNAME-SOMETHING"
 Include {
  Options {
   signature = MD5
   Exclude = yes
   IgnoreCase = yes
   # Exclude Mozilla-based programs' file caches
   WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache"
   WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache.Trash"
   WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/ImapMail"

   # Exclude user's registry files - they're always in use anyway.
   WildFile = "[A-Z]:/Documents and Settings/*/Local Settings/Application Data/Microsoft/Windows/usrclass.*"
   WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"

   # Exclude directories full of lots and lots of useless little files
   WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
   WildDir = "[A-Z]:/Documents and Settings/*/Recent"
   WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
   WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
   WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary Internet Files"

   # These are always open and unable to be backed up
   
   
   # Some random bits of Windows we want to ignore
   
   # Temporary directories & files
   WildDir = "[A-Z]:/WINNT/Temp"
   WildDir = "[A-Z]:/temp"
   WildFile = "*.tmp"
   WildDir = "[A-Z]:/tmp"
   WildDir = "[A-Z]:/var/tmp"

   # Recycle bins
   WildDir = "[A-Z]:/RECYCLER"

   # Swap files
   

   # These are programs and are easier to reinstall than restore from
   # backup
   }
   
   #these are the files to back up
   
   File = "D:/SOMETHING/PATH"
   File = "E:/SOMETHINGELSE/PATH"
 } 

}
  • Edit the comment to match your server and job info
  • Edit the Name field to match exactly what you specified in the JobDefs
  • Edit the File field at the bottom accordingly.
    1. You can specify more than one File path (recursive by default)
      1. NOTE: because this is running on a Linux system you must use / (forward slash) for it to understand slashes.
      2. You can use spaces in the path as long as the (required) quotes remain intact
      3. For Linux paths, use a root relative path e.g. /SOMETHING/PATH

[hr top=”0″ bottom=”18″ /]

Client File Services Connector (if not already serving files, see Adding a new FileDaemon):

# SERVERNAME Server
Client {
  Name = SERVERNAME-fd
  Address = SERVERNAME.YOURFQDN.com
  FDPort = 9102
  Catalog = MyCatalog
  Password = "XxXXXXxXXxXXXxxxXXXxxxXXxXX"      # password for FileDaemon
  File Retention = 7 days            # one week
  Job Retention = 7 days           # one week
  AutoPrune = no                    # Prune expired Jobs/Files
}
  • Edit the comment to match the server name
  • Edit the Name field to exactly what’s specified in the FD config.
  • Edit the Address field to the IP address or FQDN of the server hosting the FD
  • Match the Password field to the FD for that server.

[hr top=”0″ bottom=”18″ /]

Apply Changes

After modification of these core config files you must reload bacula director to detect changes:

#sudo service bacula-dir restart

If you modified FileDaemon configuration as well (in bacula-fd.conf) you must also restart the FD service:

#sudo service bacula-fd restart

[hr top=”0″ bottom=”18″ /]

Verify Changes

Enter bacula console to verify and test new configuration:

#bconsole

Bacula will respond with a console prompt:

Connecting to Director bacula.YOURFQDN.com:9101
1000 OK: 1 bacula-dir Version: 7.0.5 (28 July 2014)
Enter a period to cancel a command.
*

Check the status of your new backup set:

* run

Bacula responds with a console prompt:

Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
A job name must be specified.
The defined Job resources are:
1: Backup Self
2: Backup FINANCE share
3: Backup HR share
4: Backup Accounting share
5: Backup CLIENTS share
6: Backup IT share
7: Backup Secure share
8: Backup Media share
9: Backup Secret Admin share
10: Backup SOMETHING share
11: BackupCatalog
12: RestoreFiles
Select Job resource (1-12):

Select your new set by entering the number (e.g. 10)

Run Backup job
JobName:          Backup SOMETHING share
Level:            Incremental
Client:           SERVERNAME-fd
FileSet:          SERVERNAME-SOMETHING
Pool:             BaculaPool (From Job resource)
Storage:          Storage01 (From Job resource)
When:             2015-10-14 09:15:07
Priority: 10
OK to run? (yes/mod/no):

Enter “yes”

Job queued. JobId=6258
You have messages.
*

Check status by either waiting for email confirmation of success/fail to your bacula report email address OR by entering this console command:

* status dir

Director status will show running and scheduled jobs. If error conditions exist please consult bacula documentation.