FoxyRoxy - Older News - DevRel_02 (archive)

Tips and Techniques on day to day stuff

Re: My thoughts on a few things

Postby SBP » Sun Sep 02, 2012 7:59 pm

Hi jvb

Thank you for sharing your view and ideas.
I hope you take what I wrote in a positive way. I was only trying to do some kind of brainstorming - which might let you get new ideas on how you could let FoxyRoxy install Debian packages in a way that might ease your way, so that we (the users) might be able to handle the packages more by ourselves.
Steen
SBP
 
Posts: 74
Joined: Thu Aug 16, 2012 5:27 am
Location: Denmark

Re: My thoughts on a few things

Postby jbv » Sun Sep 02, 2012 9:55 pm

jbv wrote: I also want to carve out all of the /etc/init.d stuff and the other directories/files associated with system start-up
Current thinking is that this stuff should also live in 05-FoxyConfig.squashfs

SBP wrote:This is fine. However, I don't quite understand why we have to do that - which problems have you met that needed special attention. You wrote that the LMS was a problem, but what excactly was the problem with LMS and the init.d stuff?


Hi Steen,

This has been on my mind from the moment I started playing with "live" linux' sometime last year. I quickly realized that while overlaid container files could be great, they also have the potential to be a huge problem. I've "been there" and "done that" in the past. While I've never used them, because I just didn't believe in the technique/principle and saw the potential for problems later, it was/is a technique that was used many years ago in databases (and still is in things like SQL databases).

Take a heap of files, shove them all into one container file and you use fewer file-handles and other stuff, but you need more pointers internally and you've got to do lower-level maintenance that the operating system would normally do. For every advantage, there is almost always a disadvantage. In a database it isn't so bad because the very nature of what you're doing is managing data, so you can always keep some extra pointers in the container file to manage/manipulate stuff, and the file is somewhat "Straight".

However with a "live" linux, we now have two added complexities. The first is that the container file is read-only, and this is good thing - trust me, if it weren't, things would get real ugly, real quick. The second which his the doozey, is that it is possible to overlay the container files so the system sees them as one merged container. Add the wizardry that "cow" provides and things now start to get very tricky. The biggest single problem is the deletion of a file that is physically located inside of a container file, and there are two aspect to this problem.

While the system is running, "cow" takes care of this through the use of special "whiteout" files that only it knows about. However, when you restart the system the internal tracking system that "cow" was using is lost forever, gone, vamoose-caboose, history. So, if you really want/need to delete the file from the container file you, now have two problems...

First, how do you delete the file from a read-only container ? ... tip ... you can't
Secondly, which container file does it come from ? ... remember, there could be more than one. This is something that not even "cow" knows about, because it doesn't need to.

The only way to remove a file from a container file is to unpack the container file, delete the physical file from your working area and then repack the container file, and replace the old/current container file with the one you have just made.

This is why I have created all of the xx-load xx-save scripts. These let you manually load (unpack) a container file. Make the changes you want/need and then re-pack it. While being far from ideal for a person who doesn't really understand stuff or want to play at this level, they do go a very long way to helping someone who does.

The 99-snap script takes this to another level, as it is smart enough to work out that a file within the one container file it is working on has changed, been deleted or added, therefore it can actually update and/or remove a file from the container file. But, it can only do this because it is working with a relatively small, known set of files within known locations. I'm really quite proud of 99-snap.

I don't know if any other "live" linux has anything like it, but I think you will find that what it does would blow the mind of a lot of people once they really understand what it does. Then again, it might not, but 'ke, I'm still chuffed with it :)

So, back to why I want to strip-out the start-up stuff and put it in 05-FoxyConfig, and how LMS brought this to the fore:

The moment you install a package that adds itself (or a service) to the system start-up sequence, all of the "init" files are regenerated. This results in a veritable truckload of files being added/deleted/changed. In a typical HDD based system, this isn't a problem as the files are actually changed/deleted/whatever so on the next restart everything is sweet. But in a "live" system, this can not and does not occur. In probably 90% of cases, simply doing a 95-create or 95-refresh will result in a system that will restart and if you are lucky, it will restart correctly. FWIW: LMS would have run perfectly if it weren't for the permissions thing which is totally unrelated and solved anyway. However, things can start to get pretty messy. You can and will often have files left hanging around that shouldn't be there. While they may not be creating an issue now, they might wreak havoc after you've installed package number 4 or made tweak number 6, by which time things will be in such a mess that you won't have any idea what belongs where or what something was supposed to be doing. It is a real nightmare in the making.

Now, suppose you want to turn a service off --- How are you going to do that?

The only way would be to regenerate all of the "init" files, then start unpacking each and every container file and modifying the "init" files in each one and hope that when they all reloaded and overlaid each other, they did it properly. Add to this the fact that at the moment we only have 2 main files, yet combined they are over 300Mb and things start to take a fair amount of time.

At first I thought the answer to this problem was "re-mastering".
I think that is what a lot if not all other "live" distros do, but I'm not sure.
If it is, then this is like using a sledge hammer to put in a thumb-tack.

After giving it a great deal of thought, I have come to think that Re-Mastering is not the right thing to do. One of the reasons for this is that if I understand the re-mastering process properly, it would mean that you effectively take everything that is on the system and combine it all into one single container file. So, this means that we are basically performing a merge of all our container files and then chucking in all of our changes. Sounds like a great idea, in theory, but I intend to keep the 2 core sqfs for as long as possible. - Different reasons that may/will be explained another time.

I think there is a much more elegant solution that will be easier to implement, maintain and use. Weather it is the "correct" solution or not is probably open to debate, although that would be a debate I won't participate in.

My solution to this issue is to only have one set of "init" files and have them in a known location. Doing this, we can control where, and when they are overlaid in the whole boot-up sequence, and we know where things will live so we can modify them. If the container file is somewhat small, which it should be, the whole process will be blindingly fast (well, relatively blindingly fast). More importantly, we can pretty well automate the cleanup/fixup and if things go wrong we should be able to sort it out by hand without to much grief.

You would still be able to use the 95-series of scripts while playing/testing/shaping stuff, so none of that would change.

In summary, I think this is the right thing to do.
Therefore, I'm going to try to do it :)

Cheers
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Re: My thoughts on a few things

Postby KazzaMozz » Mon Sep 03, 2012 3:28 am

Hi Jbv
re:
I think there is a much more elegant solution that will be easier to implement, maintain and use. Weather it is the "correct" solution or not is probably open to debate, although that would be a debate I won't participate in.

My solution to this issue is to only have one set of "init" files and have them in a known location. Doing this, we can control where, and when they are overlaid in the whole boot-up sequence, and we know where things will live so we can modify them. If the container file is somewhat small, which it should be, the whole process will be blindingly fast (well, relatively blindingly fast). More importantly, we can pretty well automate the cleanup/fixup and if things go wrong we should be able to sort it out by hand without to much grief.

You would still be able to use the 95-series of scripts while playing/testing/shaping stuff, so none of that would change.

In summary, I think this is the right thing to do.
Therefore, I'm going to try to do it :)


I think you are on the right track here. Being able to still use the 95 Scripts is vital in my humble opinion. Having a known location that we can control sounds good right now. It's your puppy/baby and whatever you choose I'm happy. It's your creation after all and so far this is still the closest available for what I want to achieve and wherever it goes I'm sure it will end up way better than anything else out there.
Yes it is already starting to get a bit messy, this is inevitable when others like myself get involved. This can be good and bad. Brings to the table a few more dishes ideas etc. You either choose to taste them and say that's not for me right now or go ahead with them like the XBMC and LMS. Already I can see you are very open to new ideas and not a closed shop. I respect what it is you are trying to achieve here and am very grateful you have allowed others to share and participate in the development of your distro.
I don't know if any other "live" linux has anything like it, but I think you will find that what it does would blow the mind of a lot of people once they really understand what it does. Then again, it might not, but 'ke, I'm still chuffed with it :)


Have to agree with you totally with the above. No-one has and for those of you that say problems okay so it's not perfect but it's a work in progress and already WOW so to those who don't like it buggar off is all I can say. Find something that is perfect and does the same. It doesn't exist. I know I've looked this rocks 8-)

Keep going :ugeek:

Today I'm thinking outside the square (as I do) if it works I'll let you know if it doesn't who cares it's my time and my fun :twisted: .............finally have some peace no-one is home to distract me :lol:

Cheers
User avatar
KazzaMozz
 
Posts: 332
Joined: Tue Aug 21, 2012 9:59 pm
Location: Australia

Re: My thoughts on a few things

Postby jbv » Mon Sep 03, 2012 11:24 am

I've updated/changed the first thing that many people will see when looking at FoxyRoxy.
The first message in the Download area. To save you all re-reading the whole thing, the change made was:

Hi,

My name is Brenton, welcome and thanks for stopping by. There are a few things that we need to get out of the way, so before you do download the file below, I do ask that you read the entire contents of this message. I also ask that you take a moment to browse this forum and once you have done that, I hope you will understand what we are trying to do here. While some things may seem frivolous and other serious, that is what this is about. We are serious about the stuff that needs to be serious, yet we try to do what needs to be done in a light-hearted and friendly manner. So, lets get the ugly and serious stuff out of the way first.


I did so for a few reasons.

Firstly, I sort of overlooked the fact that I hadn't introduced myself to you all properly. :oops:

BTW, my name is Brenton - Hi :)

As you've all read this far, I figure that we are now kind-of kindred spirits. The name thing wasn't really intentional, it just happened. The longer it went on for, the more uncomfortable I felt and the harder it was to "back-out", so it was time to put a stop to it. You can address me as either jbv, or Brenton, I don't really mind and I understand that the young'ns will always read what is the reply section and use jbv, so while I may prefer Brenton, I can and will cope :)

Personally, it doesn't bother me if you would prefer to use a "screen-name" or your real-name, for whatever reason.
If you would prefer to only use a personal name only in PM's I'm fine with that too, and will respect it - no problem.

All ask is that this place remains a comfortable site to visit, and freely exchange information about FoxyRoxy.
Oh, that and the "golden rule", of which I know there are about "eleventy-nine-billion" of, but they're all "golden".

Cheers, Brenton
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Re: FoxyRoxy - Current News

Postby KazzaMozz » Mon Sep 03, 2012 11:49 am

Hi Brenton
so glad I can use your name publicly now :lol: have used the Jbv as that is the protocol if you wanted to be private. You already know who I am so
here's to the future.
Cheers
Kazza
User avatar
KazzaMozz
 
Posts: 332
Joined: Tue Aug 21, 2012 9:59 pm
Location: Australia

Re: FoxyRoxy - Current News

Postby saintless » Mon Sep 03, 2012 12:47 pm

I also prefer to address you as Brenton.

For the rest of the users my real name is Toni.
Feel free to use it instead of saintless if you like. I do not mind both ways.

Cheers, Toni
User avatar
saintless
 
Posts: 246
Joined: Sat Jul 14, 2012 7:01 am
Location: Bulgaria

Re: My thoughts on a few things

Postby jbv » Tue Sep 04, 2012 11:03 am

Darn I hate it when a "plan" doesn't come together.

I've outsmarted myself. I had actually built the latest 1.0.25 ALSA drivers (for 486/686) a few nights ago. They have been tested and it has been confirmed that they have sorted the sound driver issue which will let us get a DevRel_03 ISO built. Tonight, I made the BigMem drivers tonight and compared to the others, they came out HUGE. I checked everything I could, and decided that while the physical size on disk was 25Mb compared to 2.2Mb, they worked and did not consume any more memory while working, so that must be right ... although only God knows why, 'coz I havent' got the faintest idea :?

So, I set about making a test version of DevRel_03 - wham, splat, not-quite-bucko... While making all of this for testing ... The way I built the drivers they lived in an "update" directory and not the real "sound" directory. I thought I could just move them out of "update" and make a few "tweaks".

Well, that did not work as planned :oops:

After a few hours messing around to see if I could correct this, it seems that when playing with kernel-level drivers, you must do things properly and can't take any short-cuts or try to "cheat".

What this means is that DevRel_03 won't go into pre-beta tonight, as I had "secretly" hoped it would.

Although the way the testing goes with FoxyRoxy, this means that my "first-cut" needs to be properly tested and that takes a day or two on it's own. After this is confirmed as being "okay", I will then make the final-cut and it will be retested before it is made available.

Cheers
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

FoxyRoxy - DevRel_03 in beta-test

Postby jbv » Wed Sep 05, 2012 12:58 pm

DevRel_03.iso has entered the final stage of testing.

There are 2 issues in the existing beta.iso and both of them are Dumb oversights on my part.
I forgot to upgrade Firefox to v15 and I incorrectly renamed a link.
These will be rectified (and tested) before final release which will most likely be this weekend.

The changes/fixes/enhancements are:

1) ALSA drivers have been updated to 1.0.25 for all kernels.
- There is no need for any audio patching <fx:yippee>

2) Unetbootin is included as a standard package

Changes to Scripts

1) All 95-series scripts now preserve permissions

2) All 95-series scripts and any script that saves/writes now flushes the disk buffers before completion and all flushing messages are consistent.

3) 95-create and 95-refresh have a setting to perform a "QuickClean"
- This removes all .wh. and .bak and .log files from 95-snap
- The default for "QuickClean" is "yes"

99-snap now creates/updates 05-FoxyConfig
05-load = new script
05-save = new script

05-saveservices = new script that updates 05-FoxyConfig with the current system services startup sequence
Note: this is only visible after /scripts/special/05-punchservices has been run (See below)
This script should be considered as being in a extremely beta phase.
It has not undergone much testing, so please be careful and expect it to mess up until it is perfected.
It is really here more for my testing at this point in time, although I would appreciate feedback on any testing you may be game enough to undertake.
Is now a good time to remind everyone that this is a Developers Release and as such, you are expected to be a bit of a developer ? :lol:

General note about 99-snap I understand that it could be argued how this should be named 05-snap
My thinking is that this is a very special script that you are likely to use a bit, yet we do not want to encourage overuse of it.
To me, it is the last thing you do. Hence it is worthy of the 99- monika <spelling>. As I'm the boss, I win :P

[ New Scripts in /scripts/special ]
01-remove486 = Removes the 486 kernel files from 01-FoxyRoxy
01-remove686 = Removes the 686 kernel files from 01-FoxyRoxy
01-removeBigMem = Removes the 686-bigmem kernel files from 01-FoxyRoxy

05-punchservices = This one is a doozey
It removes all startup files from 01-FoxyRoxy and 02-FoxyDesktop and punches[/m] them into 05-FoxyConfig
It also [i]punches
05-saveservices into /scripts and removes itself and 05-saveservices from /scripts/special

[ Other Changes ]
/root/.profile has been tweaked. When logging into tty1 it now looks to see if the NVIDIA drivers are loaded before auto-starting X
This is a bit of a kludge but it means we don't need to patch 02-FoxyDesktop to use the NVIDIA drivers.

NVIDIA Nirvana ... The NVIDIA drivers have now been pre-integrated for all kernels.
The drivers are now included in the ISO with the filename of 011-FoxyNVIDIA.squashfs.noload
To load the NVIDIA drivers, simply rename the file to be 011-FoxyNVIDIA.squashfs and restart - YEP ... That's all folks 8-)

Once loaded, special 011-nvidia scripts will auto-magically appear in /scripts
There is a merge script included. Once this script is run, you can delete the .noload file
Oh, I turned off the cursor shadow so it is no longer on by default. Sorry 'bout that. It looked neat when I first turned it on, but after a little while it started to annoy me, so it's gone :) Feel free to turn it on again if you want to.
Any changes made to the NVIDIA configuration will be saved into 05-FoxyConfig by 99-snap 8-) 8-) 8-)

miniDLNA Madness ... miniDLNA has been updated to 1.0.25 (latest version as at last week)
The miniDLNA startup script in /etc/init.d has finally been fixed to allow network cards to obtain their IP from slow DHCP servers. This can also take a little while to obtain if your network card requires firmware to be loaded (as many Realtek cards and embedded chipsets do). If the miniDLNA service is turned on at startup, you may notice a message prior to the signon prompt. The message will count from 0 to 200 (in under 20 seconds) and will stop as soon as the card has a valid IP or the timeout expires ( so headless machines won't lock up :) ) You can terminate the process by pressing a key on the keyboard, but with the keypress technique being used the spacebar and shift/control/alt/enter keys won't work - sorry, abuse the authors of BASH.
I think what we have here is the best compromise. The wait is totally dependent on factors outside our control, it doesn't take long, you get a nice clean login prompt that isn't over-written, it doesn't lock-up, you can bypass it, and it works. All things considered, we couldn't really ask for much more.

apt-get repositories have been restored to the default for Debian squeeze.
Australian mirrors have been commented out, but are still there for Australians.
The repository name for deb-multimedia has been fixed, just uncomment it, the keyring has already been generated.
If there are any known good repositories for your part of the world that you would like included, let me know.
99-snap will save your apt changes into 05-FoxyConfig without requiring you to create or refresh the 95-snap

90-build.squashfs.noload now contains development headers for all kernels.

I think there are one or two other things, that I will make notes on here as and when I remember them.

As above, this is now in final testing and should be released this weekend.

The ISO grew by a whooping 3Mb and is now about 562Mb

Some questions ...

1) Should we put the SoftMakerOffice scripts into /scripts/special to save needing to people downloading them?
- I will need to re-issue them anyway, as I really want SoftMakerOffice to load as 10-xxx or be embedded.

2) Should we put the 20-build-lms (Logitech Media Server - SlimServer) script into /scripts/special to save needing to people downloading it?

This may not quite be Heaven, but we aren't far from it....

Cheers, Brenton

[edit] - Note to self ... and comments sought.

modify /usr.share/applications/isomaster.desktop and change the settings for "Categories". This really belongs in "Utility" so that it will appear in [Taskbar Menu] --> [Accessories] instead of [Sound & Other]

Do we need a CD/DVD burning package in the basic package? Any suggestions ?

I wouldn't mind a screen-shot program that will allow us to take a screen-shot of the current X11 screen from a remote console so we can grab screenshots of the [Taskbar Menu] and [OpenBox Menu] both of which vanish when they no longer have "focus". Any suggestions?

If we include the basic SoftMaker Office scripts in the base ISO, should we have the how-to-install as a text file or PDF in the /root user home directory?

There seems to be some misunderstanding of the core differences between a "live" linux and a "desktop" linux.

In so far as a "live" linux is more like an Embedded System, that you can and will poke around until you get it right, but at the end of the day the goal is to create a "static" core than never (or hardly ever) changes. Yes, to get it right, you need to poke and prod it. You need to shove and twist it and mould it and it takes quite a bit of manual work. You will never have a 100% automated system as you do with a desktop, yet it can also save your bacon while you are poking and proding to get it right. How do we get this across? Do we even try?

Oh, the big one ... Who is going to help me sort out the documentation and clean this crap up?
- We've got some real gems hidden in some threads and I think these need to be pulled out and if not moved, at least copied into other areas where they can and will be found.
jbv
 
Posts: 600
Joined: Sat Jul 14, 2012 2:02 am
Location: Sydney, Australia

Re: FoxyRoxy - Where we are at

Postby KazzaMozz » Wed Sep 05, 2012 2:08 pm

Hi Brenton
wow wow wow sounds great,can't read it all to-night it's sounds amazing.
Great work.
On the last steps for the XBMC now.
Cheers
Kazza
User avatar
KazzaMozz
 
Posts: 332
Joined: Tue Aug 21, 2012 9:59 pm
Location: Australia

Re: FoxyRoxy - DevRel_03 in beta-test

Postby saintless » Wed Sep 05, 2012 4:50 pm

Hi, Brenton,

jbv wrote:Do we need a CD/DVD burning package in the basic package? Any suggestions ?

I vote yes. FoxyRoxy should have full multimedia + burning video options.
Maybe brasero or xfburn, They need testing and take about 35 Mb of space.

Edit after testing:
Both have almost the same interface and need almost the same HDD space (about 35 Mb).
Brasero has "Create video DVD or SVCD" option which I don't see in XFburn but the rest is the same in both.
ps-mem.py gives 13 Mb RAM used for Xfburn, and 15,8 Mb for Brasero. LXtask and System Monitor give a little bit different results but they are also in favor of Xfburn.

jbv wrote:I wouldn't mind a screen-shot program that will allow us to take a screen-shot of the current X11 screen from a remote console so we can grab screenshots of the [Taskbar Menu] and [OpenBox Menu] both of which vanish when they no longer have "focus". Any suggestions?

We can do this with Take Snapshot easy. Just use 5 or 10 seconds delay option and open the menu you need to capture.

jbv wrote:Oh, the big one ... Who is going to help me sort out the documentation and clean this crap up?


I can help with this but I wonder if my thinking will be the same as yours while copying "golden" posts here and there :) I don't think we should move them from the thread where they were posted.

About SoftmakerOffice install script - if it is included in FoxyRoxy CD it will be nice to have How-to-install guide. Maybe in the same place where the script is.

Live and Desktop linux - I guess you mean are we going to try to make FoxyRoxy working as uncompressed linux installed on the hard drive and saving every change you do in the right place as standard debian installation on the hard drive. And if you do some mistake the system is dead and you need new installation.
If this is your question I say we don't need this. What we need is to provide automated way of copying the live folder on selected partition and adding the right boot code in Grub menu list (or at least giving the right example for grub menu entry to be copied manualy).

Cheers, Toni

Edit 2: You forgot to mention the Reboot and Shutdown buttons work in DevRel_3 ;)
User avatar
saintless
 
Posts: 246
Joined: Sat Jul 14, 2012 7:01 am
Location: Bulgaria

PreviousNext

Return to Using FoxyRoxyLinux



Who is online

Users browsing this forum: No registered users and 1 guest

cron