Welcome
Welcome to refracta

You are currently viewing our boards as a guest, which gives you limited access to view most discussions and access our other features. By joining our free community, you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content, and access many other special features. In addition, registered members also see less advertisements. Registration is fast, simple, and absolutely free, so please, join our community today!

Alternative usb installation method, part 2

Stuff that has not yet gone into the official build.

Re: Alternative usb installation method, part 2

Postby dzz » Thu Dec 12, 2013 9:26 pm

I just tested refracta2usb from a live session, booted from a 32GB pendrive. It is multiboot with live-media files in subdirectories. I want to install the running live system only to a second 2GB pen.

It wants to copy the entire contents of the 32GB pen to the 2GB pen. This is neither wanted nor possible and a forced reboot was needed.

Code: Select all
image_mountpoint=/lib/live/mount/medium

rsync -av --exclude-from="$excludes_file" "$image_mountpoint"/ "$usb_mountpoint"/${target_dir}

The "live" files are actually in /lib/live/mount/medium/live/ *unless* set elsewhere using "live-media-path", in which case the cmdline must be parsed to get it.

Also need to see what happens if "findiso" is used, the mounts might be different.

More later.
dzz
 
Posts: 629
Joined: Wed Apr 27, 2011 11:53 am
Location: Devon, England

Re: Alternative usb installation method, part 2

Postby fsmithred » Thu Dec 12, 2013 10:55 pm

OK, second attempt at an answer here... (and no, I'm not having forum problems - my brain hurts.)

Could parse cmdline or else could ask.

Could have separate rsync commands for live/ and syslinux/. Or could tweak the excludes file with a file-selection window.

Or... (this one is no fun)
Could restrict this task to running from cdrom, not from live-usb.

And that ${target_dir} is useless in the code you copied. It only gets set when you're using the setup_multiboot function. I should not have put it there. That var was created so that I could re-use the copy_iso function for setting up multiboot. Oy. Do I need to change the setup_multiboot function so it allows you to add the current running system as a second system on a second thumb drive?

This will require some serious staring at the monitor.
User avatar
fsmithred
 
Posts: 1987
Joined: Wed Mar 09, 2011 9:13 pm

Re: Alternative usb installation method, part 2

Postby dzz » Fri Dec 13, 2013 1:12 am

Firstly, you have done a marvellous job of this. What I tested so far (most of it) is all good.

Accepting restrictions is indeed no fun but perfection sometimes takes a bit longer.

I looked again at where I left off with snapshot2usb. This function (maybe untested, incomplete and needing adaption to suit r2u) might help:

Code: Select all
set_livemediamounts () {

# this function is only for a live session
# the image is already mounted

# live-boot (3.0~b4-1) swiched mountpoints to /lib/live
# live-boot (3.0~b7-1) Updated file paths for /lib/live/mount  http://lists.debian.org/debian-live/2012/10/msg00021.html
# live-boot (3.0~b7-1) Renamed /lib/live/mount/image to /lib/live/mount/medium  http://lists.debian.org/debian-live/2012/10/msg00023.html

# wheezy is (3.0~a35-1) 21/11/2012

SYSTEM_LIVE_MEDIA_PATH=$(cat /proc/cmdline|grep -o "live-media-path=.*"|sed 's: .*::'|sed 's:live-media-path=/::')

if [ -z "$SYSTEM_LIVE_MEDIA_PATH" ]; then
SYSTEM_LIVE_MEDIA_PATH=live
fi

# wheezy (live-boot <3.0~b4-1)
elif [ -f /live/image/live/filesystem.squashfs ]; then
LIVEMEDIAMOUNT=/live/image
FILESYSTEM=$LIVEMEDIAMOUNT/${SYSTEM_LIVE_MEDIA_PATH}/filesystem.squashfs

# sid findiso
elif [ -f /lib/live/mount/medium/live/filesystem.squashfs ]; then
LIVEMEDIAMOUNT=/lib/live/mount/medium
FILESYSTEM=$LIVEMEDIAMOUNT/live/filesystem.squashfs

# sid squashboot
elif [ -f /lib/live/mount/medium/$SYSTEM_LIVE_MEDIA_PATH/filesystem.squashfs ]; then
LIVEMEDIAMOUNT=/lib/live/mount/medium/$SYSTEM_LIVE_MEDIA_PATH
FILESYSTEM=$LIVEMEDIAMOUNT/filesystem.squashfs

fi

echo "LIVEMEDIAMOUNT=$LIVEMEDIAMOUNT"
echo "FILESYSTEM=$LIVEMEDIAMOUNT/filesystem.squashfs"

if [ -z "$FILESYSTEM" ]; then
echo "Error... $FILESYSTEM not found"

fi

}

Ability to do this is only useful in situations where the only available Linux system is your pen or cdrom and you need to load a second device for someone else or for another machine.

In that case the boot device might be multiboot but the target device would probably have no Linux system at all.
dzz
 
Posts: 629
Joined: Wed Apr 27, 2011 11:53 am
Location: Devon, England

Re: Alternative usb installation method, part 2

Postby fsmithred » Fri Dec 13, 2013 4:33 pm

I'm not sure if this will work. It separates copying the syslinux folder from the live folder. If live-media-path is something other than /live, it'll copy that folder by name. If you copy the syslinux folder that you used to boot, the menu entry should be correct for the live-media-path. (I think.) Not tested - I have no suitable test box now, except when I'm willing to reboot my main rig.

(Edit: If live-media-path is other than /live, and you set copy_syslinux to TRUE, you'll end up having to manually edit the boot menu to add the live-media-path. I suppose I could add a test for that situation, and give a warning if it arises.)

Also, I think the excludes-file option can be removed. Pretty sure that only makes sense when running mkusbcrypt and you're copying your home folder.

Replace the copy_livemount function in refracta2usb (around line 300) with the following.
Code: Select all
# Copy system from the running live system
# If system is running from a custom live-media-path, copy from there.
copy_livemount () {


   live_dir=$(grep -o "live-media-path=.*" /proc/cmdline |sed 's: .*::' |sed 's:live-media-path=/::')
   if [[ -z "$live_dir" ]] ; then
      live_dir="/live"
   fi

   image_size=$(du -sh "$image_mountpoint"/${live_dir})
   check_size

   rsync -av --exclude-from="$excludes_file" "$image_mountpoint"/${live_dir} "$usb_mountpoint"
   if [[ -e "$image_mountpoint"/isolinux ]] ; then
      if [[ $copy_syslinux = "TRUE" ]] ; then
         rsync -av "$image_mountpoint"/isolinux "$usb_mountpoint"
         mv "$usb_mountpoint"/isolinux "$usb_mountpoint"/isolinux-$(date +%m%d%H%M)
         rsync -av /usr/lib/refracta2usb/syslinux "$usb_mountpoint"   
      else
         rsync -av "$image_mountpoint"/isolinux "$usb_mountpoint"
         isolinux2syslinux
      fi
   elif
      [[ -e "$image_mountpoint"/syslinux ]] ; then
         if [[ $copy_syslinux = "TRUE" ]] ; then
            rsync -av "$image_mountpoint"/syslinux "$usb_mountpoint"
            mv "$usb_mountpoint"/syslinux "$usb_mountpoint"/syslinux-$(date +%m%d%H%M)
            rsync -av /usr/lib/refracta2usb/syslinux "$usb_mountpoint"
         else
            rsync -av "$image_mountpoint"/syslinux "$usb_mountpoint"
         fi
   fi      

   if [[ $add_hooks = "TRUE" ]] ; then
      echo " Adding Refracta custom hooks..."
      rsync -av /usr/lib/refracta2usb/hooks "$usb_mountpoint"/${live_dir}
   fi
   finished_message="Live image copied."
   finished_dialog
}


Edit2: Added missing quote on [[ -e "$image_mountpoint"/syslinux ]] right after 'elif'.
Edit3: Added missing $ on live_dir=$(grep -o blah..., and moved image_size and check_size down below live_dir=
User avatar
fsmithred
 
Posts: 1987
Joined: Wed Mar 09, 2011 9:13 pm

Re: Alternative usb installation method, part 2

Postby thwak » Sat Dec 14, 2013 9:30 pm

I think the excludes-file option can be removed.
Pretty sure that only makes sense when running mkusbcrypt and you're copying your home folder.

To me, the only "doesn't make sense to consider excludes" case is "Create a live-USB using files taken from a live-CD .iso file"

The 'r2u_exclude.list' seems incomplete, compared to refractasnapshot
(content of /tmp dir is copied, machine-id is copied, etc)

By the way, throughout the helpfiles... "excludes" is never mentioned.
I believe it's important for the user to:
1) be aware of what will be excluded
2) consider other desirable entries for his excludes list + which entries in the default list are undesirable

The refracta2usb script specifies 'monaco' xterm font. Is that font even installed?

The refracta2usb gui script opens a too-narrow-for-the-content dialog window and the devicename + path lines become wordwrapped.

possible confusion:
"Run refracta2usb from the System menu or from command line:"
vs
# root should NOT run this script (ref: refracta2usb line 121)(version="refracta2usb 0.9.4e 2013-12-02")
vs
"Edit the variables in etc/refracta2usb.conf" (ref: help_r2u.txt)
thwak
 
Posts: 165
Joined: Tue Nov 20, 2012 3:58 am

Re: Alternative usb installation method, part 2

Postby fsmithred » Sun Dec 15, 2013 11:47 am

thwak wrote:To me, the only "doesn't make sense to consider excludes" case is "Create a live-USB using files taken from a live-CD .iso file"

Or cases where files are taken from the mounted contents of a live-CD .iso file. (e.g. copying from /lib/live/mount/medium). Those two situations cover everything except mkusbcrypt.

The 'r2u_exclude.list' seems incomplete, compared to refractasnapshot
(content of /tmp dir is copied, machine-id is copied, etc)

Exactly what did you do? The /tmp of the running system is not copied.

By the way, throughout the helpfiles... "excludes" is never mentioned.

Good point. It should be mentioned in the section explaining mkusbcrypt.

The refracta2usb script specifies 'monaco' xterm font. Is that font even installed?

Probably not. Replacing monaco with mono or dejavu looks exactly the same on my system. Replacing it with sans or serif looks bad. Not specifying the font at all seems to disable the font size setting. I used a larger font in terminals that might require user input, such as a passphrase for encryption.

The refracta2usb gui script opens a too-narrow-for-the-content dialog window and the devicename + path lines become wordwrapped.

Where are you seeing this? And which version? I'm not getting that with zenity or yad. I have seen that happen when there's been a typographical error in the code for the window.

possible confusion:
"Run refracta2usb from the System menu or from command line:"
vs
# root should NOT run this script (ref: refracta2usb line 121)(version="refracta2usb 0.9.4e 2013-12-02")
vs
"Edit the variables in etc/refracta2usb.conf" (ref: help_r2u.txt)

Thanks. I do plan to clean up the documentation, and it helps to know what parts are confusing for other people.

Here's a worse problem that I noticed - some zenity windows don't like to have the buttons defined.
Code: Select all
zenity --file-selection --file-filter="*.[iI][sS][oO]" --file-filter="*" --width=640 --height=640 --title=$"Select Live ISO" --text="Select the CD image file."--filename="" --${BUTTON0}="OK"${BUTTON0NUM} --${BUTTON1}="Quit Task"${BUTTON1NUM}
--ok-label is not supported for this dialog
Running mount-loop using zenity resulted in /media being deleted. For now, I removed all the button definitions in the file-selection windows.

And selecting a partition with a zenity window fails:
Code: Select all
:~$ device="/dev/sde"
:~$ find /dev -mindepth 1 -maxdepth 1  -name "*[sh]d[a-z][1-9]" | grep $device | sort | awk '{print "\n" $0 }' | zenity --list --title="/home partition" --text="Select a partition for the encrypted home." --multiple --column ' ' --column 'Partitions' --height=380 --width=150


(zenity:27752): GLib-WARNING **: /tmp/buildd/glib2.0-2.33.12+really2.32.4/./glib/giounix.c:411Error while getting flags for FD: Bad file descriptor


I found /var/lib/dbus/machine-id. Anywhere else it's stored? That file needs to be added to the refractasnapshot excludes list. Edit: No, I don't need to add it - it's already there. That file is not present inside the filesystem.squashfs (checked refracta-7.2_i386 iso).
User avatar
fsmithred
 
Posts: 1987
Joined: Wed Mar 09, 2011 9:13 pm

Re: Alternative usb installation method, part 2

Postby thwak » Mon Dec 16, 2013 12:35 am

machine-id:
I wasn't aware of its existence until dzz mentioned it here in the refracta forum
sid-refracta-and-refracta-tools-t292.html
and, from my reading, seems like its path may vary (across debian releases? across the various debian-based distros?)
so the post-install routine of dpkg looks for /var/lib/dbus/machine-id or /etc/machine-id

The answer to "Anywhere else it's stored?" should be "no". However, some installed package may have, in the spirit of backward compatibility, created a symlink (probably in the etc dir, pointing to the var/lib/dbus path)

Do we really need machine-id? (Yes. d-bus expects/demands it)
Doesn't its presence represent a privacy/fingerprinting issue? (i worry that it does, but except for a few workaround-ers posting to Gentoo forums, few people seem to care)

After excluding machine-id, per dzz's October post, I didn't encounter any errors when booting a snapshot...
...yet (but) I don't recall finding a call to "dbus-uuidgen" (creates machine-id, if missing) within the refracta init scripts.

(zenity:27752): GLib-WARNING **: /tmp/buildd/glib2.0-2.33.12+really
Really? Yeah, really. Howabout just dump the ongoing headache of coding fallbacks to accommodate zenity.
bloat-alert-zenity-vs-yad-wheezy-t148-20.html

The refracta2usb gui script opens a too-narrow-for-the-content dialog window and the devicename + path lines become wordwrapped.

Where are you seeing this? And which version? I'm not getting that with zenity or yad.

Where == dzz's Dec10 refracta sid, which contains "refracta2usb 0.9.4e 2013-12-02"

Not specifying the font at all seems to disable the font size setting
I never considered that. Thanks for the insight.

Exactly what did you do? The /tmp of the running system is not copied

Do? Nothing.
My comment might (should) have been:

From examining the refracta2usb script along with the r2u_exclude.list
...it's not apparent that the script already, by default, excludes the /tmp path.
Suggestion: add documentation to clarify (?inline comment within r2u_exclude.list)

# refracta2usb 0.9.4e 2013-12-02
# . . .
# Copy system from the running live system
copy_livemount () {
whoosh, I still fail to understand how the script manages to exclude the tmp path
thwak
 
Posts: 165
Joined: Tue Nov 20, 2012 3:58 am

Re: Alternative usb installation method, part 2

Postby fsmithred » Mon Dec 16, 2013 3:43 am

It doesn't need to exclude /tmp, because it doesn't copy it. "Copy system from the running live system" is misleading. It does not copy the running system - it copies the image the running system booted from. If you've made changes after booting, they won't be copied. If you made changes that are stored in a persistent file or partition, those changes won't be copied.

So yeah, that needs to be made more clear in the documentation, especially since one of the big features of the installer is that it copies all your changes. If you want your changes on a live usb to be copied, you need to create a snapshot from it first, and then use that to make a live-usb.

I do recall seeing a narrow window and screwed up columns, and I think I fixed them. Try this one - copy it to /usr/bin/refracta2usb - http://distro.ibiblio.org/refracta/file ... cta2usb.sh

And don't use refracta2usb with zenity right now. Yeah, I'd like to get rid of it, but the fact that it's in the debian repo and some people might not want to install yad, keeps me keeping zenity.
User avatar
fsmithred
 
Posts: 1987
Joined: Wed Mar 09, 2011 9:13 pm

Re: Alternative usb installation method, part 2

Postby dzz » Mon Dec 16, 2013 2:48 pm

The only reason I know why Yad is not in official Debian is no DD has yet volunteered to maintain it.

The author (who does not even use Debian) forked because Zenity was (is) apparently not actively developed beyond bug-fixes, language support and gtk3 compatibility.

I can't help much with testing something I see as deprecated as I have already limited time to test what I want to, enjoy doing, will actually use, learn from and hopefully make a some contrbution to positive progress of.
dzz
 
Posts: 629
Joined: Wed Apr 27, 2011 11:53 am
Location: Devon, England

Re: Alternative usb installation method, part 2

Postby fsmithred » Mon Dec 16, 2013 3:07 pm

I think I fixed the problems with zenity. The list windows for selecting partitions needed '--radiolist' or they did not output the selected partition. This resulted in /media being deleted in (I think) the unmount-loop script. Also, zenity failed if the file-selection window had custom buttons. There are now new variables in the yad-zenity compatibility function, so that zenity gets the radiolist (but yad does not) and yad gets the custom buttons (but zenity does not.)

http://distro.ibiblio.org/refracta/file ... 0.9.4g.deb

Code: Select all
refracta2usb (0.9.4f) unstable; urgency=low

  * Moved all error logs to ~/.refracta2usb.
  * Detect user id for ownership of files made by root scripts and
  * filesystem in non-persistent loopback files.
  * mkloopback: Grouped code into new function - setup_boot_menu.
  * mkloopback: Detect live session to locate boot menu and initrd.
  * mkloopback: Fixed bug that left $PERSISTENCE null.
  * Removed custom buttons from zenity file-selection windows.
  * Added --radiolist to zenity lists, otherwise, no output.
 

-- fsmithred <fsmithred@gmail.com>  Mon, 16 Dec 2013 09:55:00 -0500


Edit: updated it again. 0.9.4g (Added a couple of tests to make sure selected device exists and is a block device.)
User avatar
fsmithred
 
Posts: 1987
Joined: Wed Mar 09, 2011 9:13 pm

PreviousNext

Return to Experimental

Who is online

Users browsing this forum: No registered users and 0 guests

cron
suspicion-preferred