Merge packaging
Overview
With merge packaging you merge your scripts and files into a copy of Shoes and create a completely new Application with no apparent ties to Shoes (some light work is required from you). The other styles of packaging installs Shoes and then runs your script.
How it works :
You can look inside the lib/package directory of Shoes to read the ruby scripts. They follow a general pattern.
- Copy Shoes (and it's Ruby)and any desired gems. Then copy your files to top. Merge! Then Magic and then call a Module/script to wrap that into an installer.
About the magic: The normal lib/shoes.rb is replaced with a minimal shoes.rb script that runs your starting script and then it deletes some things your end users can't use like the Shoes manual.
- yaml files drive everything (or they should if you're serious).
- command line scripts read the yaml and call the moodule with the magic inside. You don't have to write these tiny scripts but you should. See the links below and read the pages for the details.
- GUI to build the Yaml. You can reach the GUI builder from the Cobbler menus. It a shiny way to create the yaml file for you- remember to save it! Yes it will call the magic Module for you if you 'deploy' but thats just incidental.
Project Setup
Source directory layout
A wise developer likes to put all of an applications files in groups of directories (Folders) with just one at the Top - "foobar-project" for example. If you are used to Ruby standards you'd probably have a lib directory and a test directory. The problem is that Shoes has a lib and a static folder that's going to get copied and you really, really do not want to replace one of those because you named a file the same. The simple way is too look at what's in Shoes lib directory and pick a different name for your files in your lib directory. The static folder is less problematic if there is a collision but don't tempt fate. In fact you don't want your files to conflict with anything in the Top directory of Shoes. Of course, you can ignore the Ruby way and name your top level folders anything but 'lib' and 'static'. Bonus points if you think about how the 'fonts' directory can be used without changing the name.
startup.yaml
Starting in Shoes 3.3.7, Shoes (i.e. your application will read `startup.yaml'. See the Setting section of the manual and Shoes 3.3.7 . You want a startup.yaml if only to change the application name. The rest of the settings might not matter to you and that's fine, there are all optional - but they default to Shoes names a behaviours and that might matter to you.
package directory
You need another directory or folder, one that is not part of or inside your 'foobar-project`. Call it 'package-foobar' for example. That's where your .exe or .deb or .dmg will be placed. It's also a good place for those installer required background images, icons, packaging yaml files and your own packaging scripts.
For example here is my 'ytm' app directory
~/Projects/ytm$ ls
README.txt Ytm-license.txt ytm.rb ytmsec.csv.sav
TODO.txt ytm.png ytmsec.csv
Only one Shoes script, ytm.rb, a csv and some text files. It should noted that this is a lousy place to put files that your script will write too? Why? Because your user can't write to files in system space. Bad Developer! For Shame!
Here is my 'ytm-package' directory.
:~/Projects/ytm-package$ ls
dmg.sh installer-2.bmp osx-background.png win-merge.rb ytm.icns
fpm.sh lin-merge.rb osx-merge.rb win-ytm.yaml ytm.ico
installer-1.bmp lin-ytm.yaml osx-ytm.yaml Ytm.app
The Ytm.app and dmg.sh is leftover from packaging on OSX. fpm.sh is leftover from a linux packaging. The yaml files are just platform specific variations. The ruby scripts are used for command line driven packaging after the yaml is mostly correct because it's much faster for me to use them than the GUI.
Merge packaging on Windows
Command Line or GUI?
You can do both. There is a GUI wizard to help you define your app and build a configuration file. That configuration file can be used by the GUI to create the installer. Or, if you like scripting your builds, you can pass the configuration file to a tiny Ruby command line program that you create.
You can find Merge Packaging as a menu option in Cobbler. That launches the GUI. It is recommended you use the GUI to build the configuration file.
Windows GUI
Did I mention this result will install on Window 10?
This scheme requires two additional programs be installed. ResourceHacker and NSIS Unicode. The Cobbler screen will help you download and install them but you do have to tell Cobbler where you installed them and you should save that so you don't have to do it over and over.
NSIS creates the installer and if you don't customize it the installer is going to look a lot like Shoes and not like your application. Sadly NSIS has funny rules about Image formats. They have two old-time .bmp images (bmp-1) and you really want the sizes to match what is expected (or very, very close). You should have a couple of icons. One for the application that you store in your app folder and that you tell Shoes code to use app.set_window_icon_path. You also need an Icon for the installer created exe. They can be different files and probably should be. In the screen shots below the app is just one file 'isp.rb' in the dropouts folder.
I chose to save the configuration is an isp.yaml file.
Windows command line
You need to create a short ruby script and we want the yaml file we created from the Wizard. Cd to where you saved the yaml file and create win-merge.rb file. Then we use command line Shoes to the merge with `cshoes.exe --ruby cmd-merge isp.yaml
Here is win-merge.rb.
# run this with cshoes.exe --ruby win-merge.rb file.yaml
require 'fileutils'
include FileUtils
require 'yaml'
opts = YAML.load_file(ARGV[0])
opts['packdir'] = Dir.getwd
home = ENV['HOME']
#appdata = ENV['APPDATALOCAL']
appdata = "#{home}\\APPDATA\\Local" if !appdata
appdata = appdata.tr("\\",'/')
util = YAML.load_file("#{appdata}/shoes/package/util.yaml")
opts['RESH'] = util['rhp']
opts['NSIS'] = util['nsp']
GEMS_DIR = File.join(appdata, '.shoes','+gem')
puts "DIR = #{DIR}"
puts "GEMS_DIR = #{GEMS_DIR}"
require "package/merge-exe"
PackShoes::merge_exe(opts) {|t| $stderr.puts t}
The only trick here is that it knows where cobbler downloaded and installed ResourceHacker and NSIS. If you didn't do that then set the opts[] to point to where you did install them
Merge packaging on OSX
Requirements
You'll need a Macintosh running Shoes. You also need the hdiutil and possibly some more esoteric osx programs. You may need to install xCode which requires a free (cost) Apple Developer membership. I don't make the rules.
You also need a license file, some .icns or two and a background image. See the options section below.
Sadly, to run the command line packaging you need a system Ruby or a Ruby from rvm. You can't use the Ruby inside of shoes for that.
GUI
Command line
The osx-merge.rb script differs a little from the Windows and Linux variations. It needs the shoes_at option to point to where you installed Shoes and it needs the ruby version and arch settings. Get them Cobbler->Info. You also have to use a system (or rvm) ruby to run the script - ./cshoes --ruby doesn't work. Setting the ruby constant DIR is required.
# run this with ruby osx-merge.rb file.yaml
require 'fileutils'
include FileUtils
require 'yaml'
opts = YAML.load_file(ARGV[0])
opts['shoes_at'] = '/Users/ccoupe/build/yosemite/Shoes.app/Contents/MacOS'
opts['target_ruby'] = '2.3.6'
opts['target_ruby_arch'] = 'x86_64-darwin-14' # yosemite 10.10
if opts['shoes_at']
DIR = opts['shoes_at']
else
DIR = "/Applications/Shoes.app/Contents/MacOS" # EXPECT A FAIL
end
opts['packdir'] = Dir.getwd
home = ENV['HOME']
GEMS_DIR = File.join(home, '.shoes','+gem')
puts "DIR = #{DIR}"
puts "GEMS_DIR = #{GEMS_DIR}"
require "#{DIR}/lib/package/merge-osx"
PackShoes::merge_osx(opts) {|t| $stderr.puts t}
Options
Shoes uses and includes the Yoursway-create-dmg scripts which has options and defaults that Shoes doesn't exploit. To fully exploit those options, Shoes creates a dmg.sh bash script in your package directory which you can modify and run to play with. You could modify the lib/package/merge-osx.rb file but the changes would be lost in the next Shoes upgrade - so create a new issue at the wiki to report your enhancement or bug.
Merge Packaging for Linux
This is the way to build and package an application that hides Shoes-ness as much as possible. It's called Merge because it copies your app and files into a copy of Shoes and arranges to your app to start instead of shoes.rb (the splash screen). Then we create an installer for that combination. We use the AppImage format because it's much easier for you and Shoes to deal with compared to Flatpack, Snap or the older deb and rpm.
The result is a single application file that doesn't depend of Shoes being separately installed.
AppImage
It's like Flatpack or Snap, only simpler. That said, AppImage is not all that forgiving - there are rules and it's easy to break the rules and Shoes doesn't really have the correct information. Shoes will build the appimage but more importantly, Shoes provides the scripts and data for you to package without the GUI. That way you can modify the data files to fit the rules, as best you can and (re)package from the command line until you and the app store maintainer are happy. There are several common download sites for appimages or you can distribute from your own website or both.
You can use the Cobbler->Package->AppImage to build the needed files and it will attempt to package them. Once you have the yaml file from Shoes and the optional appdata.xml file you can (re)package from a simple script or you can use the GUI again. For example: my sample app, Ytm has a ytm.yaml I built with the GUI. The RUBY constant DIR points to where Shoes is on my system, quite Likely it is "#{ENV['HOME']}/.shoes/walkabout" for you. You'll also need to set the require path "#{DIR}/lib/package/merge-appimage" might work. The yaml location is your choice.
#! /usr/bin/env ruby
require 'yaml'
opts = YAML.load_file("../ytm/ytm.yaml")
DIR='/home/ccoupe/Projects/shoes3/xlin64'
require "/home/ccoupe/Projects/shoes3/lib/package/merge-appimage"
PackShoes::merge_appimage(opts) {|msg| puts msg }
If it's not obvious yet, everything is being done with ruby and the appimagetool. You can examine the ruby to see what's really going on.
The yaml file contains the required fields from the GUI wizard which you'll probably modify by hand or by GUI and you see fit. Inside the yaml is a usexml boolean and if set to true, thenappimagetool` will want an AppStream xml file. I suggest your first try should not use the AppStream XML. Get a working packaged program to packaging and then you can deal with the AppStream xml.
Menus
AppImage uses a .desktop file created by the Shoes code so the package app will show up in the users desktop menus. It's possible you or your end user or your Linux distro provides AppImage integration automatically. It's a neat thing to have but sometimes you don't want it (repeated testing is one reason - you have to delete the old menu by a right click, but there are other reasons too). The no_integrate option can be set to true in the yaml file.
AppStream AppData
You need to install the appstreamcli and appimage-util programs to process the xml appdata. These may be difficult to find. Ubuntu/Mint has them (separate packages, of course) but you have to search around to find them. On raspbian, sudo apt install appstream appstream-util worked for me.
download https://github.com/AppImage/AppImageKit/releases/download/12/appimagetool-armhf.AppImage
It's been simple so far (really!). When you get to the xml you'll have to deal with some issues you may not have thought about (and the xml is really finicky). You need a website. You shouldn't use shoes.rb or mvmanila.com because we know nothing about your app. Example.com would work but doesn't look very professional does it? Are you going to publish the appimage in a public store or repository? If so, you want it look good in the store, right? Is the icon good looking? Big enough? Memorable? You can have screen shots as part of the appimage so store browsers can peek at your app. You'll need a place to download those from (your website perhaps) appstream-util will look to see. You'll want a more useful description than 'Built by Shoes'. This one reason you really need to do the packaging by a tiny script like I showed above - lot's of testing and thinking.
older packaging method
There is a downside to all this goodness. You will have to do some more work and more importantly you can't package across platforms with merge packaging. If you want to make a Debian .deb, you need to do that on a Linux system running Shoes.
Command Line or GUI?
You will do both. There is a GUI wizard to help you define your app and build a configuration file. That configuration file can be used by the GUI to create another script that you have to run. Linux packaging uses the the fpm program.
There's a complication - Shoes won't call the fpm.sh for you. It turns out that fpm is actually a Ruby gem so you need to install it in an available Ruby, different from the ruby inside Shoes. gem install fpm That IS NOT the Ruby inside Shoes, probably your system installed ruby (or rvm/rbenv if using them) . lin-merge.rb you'll have to write for youself but we have a template down below.
build-lin.rb and merge-fpm.rb are scripts inside Shoes.
You can find Merge Packaging as a menu option in Cobbler. That launches the GUI. It is recommended you use the GUI to build the configuration file (yaml) at least once. While you can modify the fpm.sh file you probably shouldn't do it lightly.
Linux GUI
It is a very good idea to save the yaml file and reload it at the beginning to the GUI. It saves much typing.
Command line
Lets get the .deb built. The gui 'deploy' copied things to the 'packdir' directory so cd there. Mine has a fpm.sh and a Ytm-app/ directory. That -app directory has your code mixed in with a slightly modified Shoes and your desired gems.
./fpm.sh that's it. Now you have a .deb, Ytm-app.deb in this example.
You probably like to automate the build and not have to go through the GUI since nothing has changed there. Lets create the script to do it. Call it lin-merge.rb if you like.
# run this with path-to-shoes/shoes --ruby lin-merge.rb file.yaml
require 'fileutils'
include FileUtils
require 'yaml'
opts = YAML.load_file(ARGV[0])
opts['packdir'] = Dir.getwd if ! opts['packdir']
home = ENV['HOME']
GEMS_DIR = File.join(home, '.shoes','+gem')
puts "DIR = #{DIR}"
puts "GEMS_DIR = #{GEMS_DIR}"
require "package/merge-lin"
PackShoes::merge_linux(opts) {|t| $stderr.puts t}
Yes. it could be improved slightly. You can run this with your installed Shoes with a ~/.shoes/walkabout/shoes --ruby <path-to-cmd-merge.rb> <path-to-yaml> [note: if you use a different Ruby to run this then set DIR to where Shoes is]