dpkg = the unsung hero while apt-get receives all the glory

Here's a radical concept. Let's leave notes and mini-how-to's for each other

Re: dpkg = the unsung hero while apt-get receives all the gl

Postby KazzaMozz » Sat Sep 22, 2012 12:56 pm

Hi Brenton
FoxyRoxyLinux will not be a replacement for a real Linux HDD based OS.
That is not my goal. My goal is for a good, fast, stable, 100% Debian compliant OS that does not write back to the main OS media. This means that ideally, you would test a package to see that it works for you and does what you want, and then inject it into your particular build of FoxyRoxyLinux.

Exactly what I want. A set with exactly what you want and leave it alone. It will tick all the bells and whistles for what i want.

A script would not really be required. It wouldn't take much to actually take a bunch of orphan packages and wrap them up into a single sqf. If you wanted to, you could then also merge all of the dpkg.db.new files for them into a single set of dpkg.db.new files.


Yep for me this would be perfect. Especially for the server side of things. Once it's perfected to be able to wrap up nice and warm and cosy in a single sqf. Bravo.

Anyway I'm signing off early to bed. Have a 6.00am flight so we leave at 4.15am :(

Hooroo

Kazza
User avatar
KazzaMozz
 
Posts: 332
Joined: Tue Aug 21, 2012 9:59 pm
Location: Australia

Re: dpkg = the unsung hero while apt-get receives all the gl

Postby jbv » Sat Sep 22, 2012 10:15 pm

Hi Kazza,

I should have it sorted by the time you get home.
Have a great time sun'n and fun'n.

Spare an occasional thought for those of us slaving away to keep the country, and economy rolling along while you're living it up. On second thought ... Nah, don't bother ... just enjoy!

Stay safe, Brenton
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Re: dpkg = the unsung hero while apt-get receives all the gl

Postby jbv » Sun Sep 23, 2012 11:45 am

If you're still with me, it might be time to let you know where this is at.

While being slightly distracted by making 99-snap the most awe-inspiring tool known to man, I gave a little thought as to what package I would use to try and document this process and also document the process of creating the ultimate sqf injection.

Slight Detour (for context) ... During the week, I have totally re-organised my home network as I just had to many wireless devices beating up on my mode/router. This is a very ugly story and I will save you from hearing of it, but during this process, I stumbled upon a really neat program called AngryIPscanner. As I am looking to build a Home NAS/Server, I thought the Linux version of this program would be a neat addition to FoxyRoxyLinux.
However, it requires the Java Runtime-Libraries! ... <fx:eek> There is only on thing that scares any distro builder more than the Java Runtime libraries and that is the Qt run-time libraries. So, as there was still something worse, I decided to make this my "test platform". :lol:

... Yeah, I'm a chicken, and would really like to keep what little I have left of my sanity. While I do have Qt4.8 apps running on FoxyRoxy, I can't remember what I did or how I did it, and I'm not looking forward to doing it again, so if you need Qt4.8, then that is being left as a reader exercise :lol:

Anyway, back to the topic at hand: I have created a very nice injection that contains all of the required Java 6 runtimes and it includes the IcedTea add-on. If you don't know what IcedTead brings to the table, then welcome to the club, because neither do I. If you do know, then please tell us. This injection also contains the AngryIPscanner application as a "test app" that requires Java and IcedTea to run.

At this point of time, I have not done anything to clean or strip out the old (pre-existing) stuff from the dpkg.db files, but I have renamed them so they won't cause any issues on any system. The end result is that I have an almost perfect orphan application to play with.

If you are interested, you can download the sqf <from here>
EDIT: the above link has been disabled - all will become clear further on.

After renaming the .noload to be a .squashfs and restarting
To ensure that it works, look here ... [TaskBar Menu] --> [Inernet] --> [AngryIPscanner]
... Pretty neat hu?

Now run 07-load and have a look in /tmp/sqf-icedjava
I have temporarily renames the dpkg.db files with an 07- prefix and a [/i].new[/i] suffix so they will not interfere with the real dpkg.db files, and I have also left a small note for myself/you.

Now remember that at the moment all of this stuff is a total orphan, so please do not try to load and run any other Java apps or packages as you may (read probably will) find yourself in a world of pain before this is over. However, while this sqf is loaded, Java apps should now run from within Firefox too :) :) :) :)

That's it for tonight, I'm calling it quits. I may not get to revisit this until next weekend, but I hope you can see where I am going with this and I really do think that while the ride will be wild and frustrating, once I come out the other end, it will be simply mind-blowing. At the same time, I hope it looks so elegant, and simple that you will wonder what all of the fuss was about.

The 99-snap extension ended up that way, which made me quite proud.
When it looks so simple that you wonder what the fuss was about, then you know you've done it right.

Cheers, Brenton
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Re: dpkg = the unsung hero while apt-get receives all the gl

Postby KazzaMozz » Mon Sep 24, 2012 5:57 am

Hi Brenton
sitting here with a glass of chilled wine in hand on the balcony looking over the main street just relaxing.
The 99-snap extension ended up that way, which made me quite proud.
When it looks so simple that you wonder what the fuss was about, then you know you've done it right.

Me thinks you are feeling a bit chuffed right now :lol: :lol:
I shall confirm it as I am thinking this will make life much easier for all around in regards to work currently in progress.

Cheers
Kazza
User avatar
KazzaMozz
 
Posts: 332
Joined: Tue Aug 21, 2012 9:59 pm
Location: Australia

Re: dpkg = the unsung hero while apt-get receives all the gl

Postby jbv » Sun Sep 30, 2012 3:58 am

It is done!

If you downloaded the file 07-FoxyIcedJava from the link 2 messages up, I suggest you make a few minor changes to the .sqf you downloaded. The reason for this is that I ended up creating a wizard-script that works on the 95-snap file, as this is where most people will start. The wizard takes a 95-snap, cleans it up, and injects some more scripts into the 95-snap and then creates a fresh overlay file with a name that you provide - which I will refer to as xx-MyName.squashfs. The wizard then renames 95-snap.squashfs to be 95-snap.squashfs.made-XX where XX is the numerical number sequence that you defined for your sqf.

The final xx-MyName.squashfs auto-magically includes scripts to load/save itself so you can still easily tweak your sqf. while it is an overlaid file. The wizard also puts one more piece of magic into your xx-MyName.squashfs. This piece of magic is a script that will inject your .sqf into 02-FoxyDesktop and ensure that the final dpkg database is up to date and reflects all of the packages inside your sqf which after execution is part of the base FoxyRoxyLinux on your machine.

So, to test this with the sqf you downloaded above, we need to turn that back into a standard/typical 95-snap.squashfs This is pretty straight forward and best done manually. All it really involves is removing the load/save scripts I had there, and renaming the dpkg database files back to their original names, then saving everything with the proper 95-snap filename. The exact sequence of steps is as follows:

make sure you do not have a 95-snap.squashfs file

run 07-load
delete /tmp/sqf-icedjava/scripts
delete /tmp/sqf-icedjava/var/lib/dpkg/07-textfile.(notes).txt [the original of this file]
rename /tmp/sqf-icedjava/var/lib/dpkg/07-available.new to be available
rename /tmp/sqf-icedjava/var/lib/dpkg/07-status.new to be status
rename /tmp/sqf-icedjava to be sqf-snap
run 95-save

As we will potentially be modifying our dpkg.db, we should keep a pure and clean set of dpkg.db files.
The best place to do this is inside of our 02-FoxyDesktop file. The following sequence of steps will purge the "old" dpkg.db files and create a copy of the final dpkg.db which is how FoxyRoxyLinux shipped. By purging the previous backup, our new "master set" won't require any more space on disk.

Code: Select all
#
# Create a master copy of the dpgk database in 02-FoxyDesktop
#
02-load
rm -f /tmp/sqf-foxydesk/var/lib/dpkg/available-old
rm -f /tmp/sqf-foxydesk/var/lib/dpkg/status-old
cp /tmp/sqf-foxydesk/var/lib/dpkg/available /tmp/sqf-foxydesk/var/lib/dpkg/available-org
cp /tmp/sqf-foxydesk/var/lib/dpkg/status    /tmp/sqf-foxydesk/var/lib/dpkg/status-org
02-save
#
# done
#


The sqf wizard package, also contains a script to flip the dpkg.db from your "live" system to a "clean" system, and back. This allows you to create overlay and injection sqfs that should run on any FoxyRoxyLinux regardless as to what packages may or may not be on your system.

Exactly what this stuff does and how it does it, is going to take a day or two to document.
Once it has been documented, I hope everything will become clear.

While working on this I have now finally been able to document the process to create your own add-on and/or overlay/injection. While this "documentation" is actually embedded within the wizard script, in so far as the way it works. The wizard follows the model I have been using since the beginning, and is really just an automation of what I have been doing manually. Hopefully, pulling it apart and analyzing it will make the "documentation" process a little easier and clearer.

Cheers, Brenton
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Re: dpkg = the unsung hero while apt-get receives all the gl

Postby jbv » Sun Sep 30, 2012 4:16 am

On second thought...

Hold off on making any changes to the download referred to above.
Actually, it will be better if I remove the download link all together - which I have now done.

I have just done all of the above again (for the 6th time) and I will post it with the wizard script, as an example.

I will also create another script for the wizard pack that will "patch" your 02-FoxyDesktop file and create the master set of dpkg.db files.

Cheers, Brenton
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Previous

Return to Dev Notes



Who is online

Users browsing this forum: No registered users and 1 guest

cron