Tuesday 2011-09-27

Most of today was not taken up with PPC (only about 2 minutes while I rebooted the working hardware and it actually did that without crashing). Instead most of the day was working on Fedora 16 betaRC3 issues.

Last night while installing the system got stuck in anaconda going at 100% CPU but nothing happening. I figured letting it sit all night might get it to get to the next step. Instead this morning the laptop was very very very warm and the desk had a slight discolouration from where the T61 had sat on it. I then said "oh dear", and turned off the laptop for a while to let it cool down. I then added a metal laptop riser underneath it to allow for better airflow, and rebooted for another go at a reinstall. The install this time worked perfectly, but the kernel OOPS after reboot in firmware insertion. Ok... maybe letting it run 12 hours at 100% CPU wasn't a good idea.

Let the system cool down, and reboot. No OOPs and the system got first boot going. No problems so I create an account and log in to KDE. KDE starts up and gets a crash when I try to add another desktop... known bug but trying to do trace opened up a new weird behaviour, somewhere in the KDE stack I would only get updates to apps every 30 seconds or so. It wasn't like they were stuck, just that it takes either some sort of buffer to fill up with events or a trigger like moving the mouse to get the applications to draw new data. Running a gnome app was even weirder in that it would completely grey out during these times (however xfce apps would not have this happen).

Ok do a yum update and reboot the system. Oh look a new issue. The RHGB would draw the fedora icon and then stop. No disk activity, not accepting keystrokes to F1->F4, and not able to ping. Ok reboot again with rhgb out of the grub2. .. boot fails in fsck check with no shell fallback. Ok reboot the DVD into rescue mode and find that /home's superblock needs to be "fixed" (the journal on / was able to recover but for some reason systemd.e2fsck didn't seem to be able to do so.)

Alright system boots up after a system wide relabeling and we can log back into various shells. KDE still shows problem, GNOME does not but keeps giving me sadmac faces when trying to launch apps. XFCE works without issues. At this point I am not sure if the problem is with the hardware or not so I am going to run the IBM system check tools overnight to see if it comes up with anything.

In my offtime I got a haircut and read more of the CSS cookbook to catch up with 21st century HTML.

Monday 2011-09-26

Today was mostly a catchup day. I went with the wife to early vote in our municipal elections, and then went to the book-store to look for some old sci-fi books. One of the things I learned was that living with a writer means book-stores are work and not fun.. which made my slow perusal maddening for her as she wanted to take a break.

After I got back, I learned how to do customer service by Keurig. My Keurig coffee maker had stopped working on Saturday and after trying descaling and some other things figured it was a broken pump. So this morning I expected a long phone call where I would have tobeg for a replacement. Instead I was helped by a cheerful lady "named" Dana who helped me figure out that due to the high altitudes of New Mexico, my coffeemaker was pumping grains up the "pin" causing it to block up. The fix was to use a small paper clip to clean out the tube and then run some cups through. After doing that a lot of grounds came out and I got coffee.. that made me happy. Dear Keurig, please hire more people like "Dana", she made my day. [I am not sure if she was really Dana or had a cover name.. but I enjoyed talking with her while waiting for the coffee-maker to go through the canister of water.

Tasks

• Email.. lots.
• Play around with zif
• Download and install RC2

Friday 2011-09-23

Ok so I volunteered to be a rawhide wrangler. Will see what that looks to require and will start on it Monday. What I envision it to be is that for things that are broken in rawhide but are built/working in beta channels.. I would be there to build stuff that has gotten out of sink. So if 16beta has gnome-shell-3.2.0-6 and rawhide has -2 I build the -6 version and push to rawhide. The goal will be "make rawhide consumable as much as possible."

Tasks

• Try to get a vlan change for ppc11
• Brick an APC power bar. Turns on you can turn on SSL before you upload any certificates.. which means it can't be logged in anymore.
• Play around with zif. Don't brick the laptop this time :).

Thursday 2011-09-22

For some reason I woke up at 5am convinced it was Saturday. Convinced so much that I went to bed and when the wife and child asked me to get up at 7am I put on quite a whine. Well it turns out it is really Thursday, so I am off to figure out what needs to be done for the day.

Spent my free time while waiting for techs or puppet to run .. to install Fedora Beta-RC1. Found a long list of issues:

Issues with Brc1

• KDE Desktop was non installable due to digikam issues (known)
• Caused a crash by selecting and unselecting repos
• Restarted the install and chose a minimal install
• Did a "yum groupinstall kde --exclude=digicam --exclude=kipi*" and got items installed.
• Got x running and decided to switch systemd to graphical desktop. [Made sure kdm was installed
• Had the system switch to run level 5 (graphical desktop) and found I had 'bricked' the box. No X started up and the system had no gettys
• Trying to get grub2 not to boot into this level didn't work and I decided it was late and time to just reinstall
• Did a default click through install and system seems functional. Yeah something I learned from windows works... if things stop working, just reinstall

Tasks

• Got retrace02.fedoraproject.org on the net. Test day next week should be go
• IBM called with new parts.. start the dance
• Someone is doing a bumpkus load of reviews because my bugzilla mail is full
• Fedora weekly meeting
• IBM arrived and we played does this part fix the problem
• Got a working PPC board and spend evening building it.
• Sooooo close, and still stuck with needing to update blade bios

Wednesday 2011-09-21

Last night was a string of nightmares with everything from giving a PhD thesis defense but not knowing which subject to living in a trailer after a nuclear war. Fun stuff.

Tasks

• Work with IBM on PPC issue some more.
  
• Do research on PPC server options. My google grok foo rolled a 1.
  
• NIST papers long. Brain hurt.
  

Music More WCPE

Tuesday 2011-09-20

Tasks

• PPC issues
• Spend a lot of time tracking down system ownership
• Build bastion01 and work on moving it to production
• Debug various bastion issues

Music

• WCPE classical

WCPE is a wonderful OGG station so I get to listen to pretty much commercial free classical music when I want.. I just need to make sure I do a bi-yearly donation. I miss the Norwegian OGG radio stations. I was enjoying listening to the childrens radio and learning some amount of Norwegian. Its like Old English but not.

Monday 2011-09-19

Today is International "Talk like a Pirate day" but my grasp of just normal English is sub-par currently. The local allergens are in full blossom and I have gotten a bad case of the ear tubes getting blocked up... so today was mainly drinking warm liquids hoping my left ear would pop open.

A quote from an article today:

The computer is a thing, but what people want is not a thing, but to do things.

Something we computer people forget all to often :).

Tasks

• Get IBM technician into location for PPC replacement part N+1
• Lots of Email to catchup on
• And another day another dead PPC.

Music

• WCPE -classical music

Friday 2011-09-16

Spent most of the day working with various IBM Power PC support technicians. I will say that the IBM guys worked their hardest on trying to get one of our blades working. In the end, they had a fix-now board sent out and a technician there by 1pm. The board of course didn't work. Not sure why we are having so much trouble with JS12 blades but it really really is getting old.

Tasks

• Spent talking with IBM on PPC support.
• +1 Nirik's changes to VPN
• Rebuild ppc12 so it can go back in production

Music

• WCPE - Classical Music

Thursday 2011-09-15

The Microsoft Ergonomic keyboard died last night (half the keys quit working while trying to type the last blog) so I went out and got a

Tasks

• Buy a keyboard
• Try to catchup with email
• PPC hardware fails in new and horrible ways
• Fedora Weekly Infrastructure meeting

Music

• Clannad - Magical Ring
• Def Leppard - Pyromania
• Depeche Mode - Construction Time Again
• WCPE Classical Radio

Wednesday 2011-09-14

Lots of rain last night.. I slept so solidly that I didn't hear the pager go off at all. Thanks to Seth Vidal for fixing the issues we ended up with.


Learned also that getting used to vi keys (j,k) to move around in email is a bad thing in other readers.. spent the morning removing a lot of non-junk emails

Tasks

• Junk my work email accidently. Spend time fixing
• Wondered HOW THE HENRY IBM stays in business with a crappy self-service center like it has
• Pain and agony.. it must be PPC

Music

• Depeche Mode - A Broken Frame
• Duran Duran - Rio
• Fleetwood Mac - Mirage
• Heart - Private Audition
• Phil Collins - Hello, I must be going
• Rush - Signals
• Blue Oyster Cult - Tales of the Psychic Wars

Thoughts on Rawhide

Currently Fedora supports in some form or fashion 4 releases at one time:

Rawhide

Meant for the cutting edge of packages. Packages here can be broken regularly so that fixes can be done as fast as possible versus waiting til more expensive times during an adoption curve.

Release+1 (16)

A release that will be released soon. Branched from Rawhide and then made into release quality over a 6 month period. Problems and fixes are progressively worked on so that the software is the newest "stable" version.

Release (15)

The current "release" which is meant for most users to install and run. Fixes to this release are meant to be neutral. If a newer "stable" version is released it does not mean that it is put into the Current Release but only if no fixes can be made to the version that was released.

Release-1 (14)

The release which is nearing its end of life and fixes should only be done for security and critical problems

Ok using the "standard" technology adoption lifecycle, and realizing that Fedora is meant for Innovators and "Early Adopters" versus say people in the Majority curve(**).

• Rawhide should be where people can experiment and break and fix stuff quickly without affecting the main population of users.On the standard technology curve this should be the furthest out innovators and developers.
• Release+1 should be where things are solidifying and innovators are getting things ready for others to use.
• Release should be where innovations have solidified and early adopters can try to make the various parts work for their projects.
• Release-1 is where users who are not ready to move to either Release+1 or Release and may want to jump off to something more "Main Stream" like an Enterprise Linux.

The reality though is that Fedora does not seem to be able to handle this many streams at once. There have been a lot of threads about packages not getting approved for fixes in "Release" and "Release-1" that should have been, and large and long complete breaks in "Rawhide" that have not been fixed. Looking at the threads and in particular Jonathan Corbet's reply to me.

To me the original idea of "No Frozen" rawhide sounded good. I am not sure that the implementation due to the fact that developers can only focus on 1 or 2 releases at any one time. So I think we have problems that need to be addressed after this release.

1. Should Rawhide be usable by anyone? If people are using it should they expect any kind of timeframe for problems to be fixed?
2. Should we just have a way to move people from rawhide to Release+! when the alpha split occurs and stop updating rawhide til Release+1 becomes Release?
3. Should we just give up on the concept of this many releases being available and go for "Release+1, Release, Dooom"?
4. With all the other stresses should we just end Fedora on a high note, and move along to "GnomeOS-X" and "NotADesktopOS-X"?

** People who are in the majority parts of the curves prefer items where the cost to adopt is much lower and different than people who are early adopters or innovators. This is why there is such a disconnect between the "Wow this is the greatest thing since sliced bread" and "This is the most horrible change I have ever seen." groups in adoption cycles.

Tuesday 2011-09-13

Stuff
I learned the following today how to retire a branch.. I think

fedpkg clone mediawiki114
fedpkg switch-branch -l
fedpkg switch-branch origin/el4
fedpkg retire
fedpkg push

Then go into fedorapackages and retire the packages there. (Or just follow the directions at http://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life

I set up a new set of packages for x86_64 EL-6 testing. Public key is in the directory.

Tasks

• Learn how to get fedpkg to retire stuff
• Build a bunch of fedorapkgs for testing
• Had a late afternoon page fest that was weird.

Music

• WCPE
• Adele - 19
• Adele - 21
• 
  

Monday 2011-09-12

Spent the weekend trying to get my old website moved to new cloud systems. The old box was a CentOS-3 system so.. its a bit of upgrade. Yes I am so far behind the curve I had a very EOL system on the internet.

Release time must be coming up because a lot of emails on lists are the "I would rather rant on someone else's broken stuff versus working on my own broken stuff" emails. I am getting better, my discard draft folder was emptied of 30 emails reminding people that "If you haven't anything gorram nice to say, then shut the frak up."

On the nice news.. IT IS FALL. The air has that nice aroma, the leaves are turning brown (we aren't New England so its Green->Brown with some Yellow for an Aspen) and I am feeling lively as a chipmunk.

Tasks

• A lot of email.
• Play around building audacious for EPEL-6
• Update F16 box.. the box only reboots cleanly if I turn off rhgb
• Got a continual crash in rhythmbox playing WCPE so time for a new player
• Compile audacious for EL-6 is fun. Got it working.. yeah
• Kill/maim/exterminate db01.stg
• Started a thread on devel which I hope was constructive

Music

• WCPE ogg (which killed rhythmbox)

Friday 2011-09-09

Started the day off with finding out the change I made to DNS earlier broke the box earlier. sigh. On the other hand, the weather has become nice and cool. Today's high was 75 which is well inside my "I operate well" parameters. I woke up pretty focused and clear.

Tasks

• Fix resolv.conf mess
• Spent the morning learning git
• Started reading through NIST and CC documents

Music

• Robert Plant & Allison Krauss - Raising Sand
• The Police - Zenyattà Mondatta
• Phil Collins - Face Value
• Blue Oyster Cult - Fire of Unknown Origen
• Def Leppard - High and Dry
• Rush - Moving Pictures
• The Police - Ghost In the Machine

Thursday 2011-09-08

Well today got off to a late start.. too many allergens and not enough benadryl. Or too much benedryl.. and not enough wakeup juice. Not sure which. When I got up I found I broke staging...

I upgraded my systems salt to 0.9.1 and ran into a long list of issues... mainly because I didn't have Cython installed and it wants it. Then some other ones that just seemed mid-devel issues.

Tasks

• Get db02.stg into rotation
• Break staging in new and interesting ways
• Fedora Infrastructure meeting
• More playing with salt.
• Work on retrace server

Music

• Miles Davis - Complete Birth of the Cool
• Dead Can Dance - Into The Labyrinth
• Bruce Hornsby and the Range - The Way It Is
• Allison Krauss and Union Station - Paper Airplane
• Robyn Miller - Myst
• Robert Plant & Allison Krauss - Raising Sand

Books
I am currently blazing my way through "Inside the Tornado" by Geoffrey Moore.It is an interesting read and is hopefully making it better at learning how to market stuff better. The funniest part is reading it as a book written 15+ years ago. Most of the companies written are either gone or changed greatly due to market changes. It is memory lane remembering the whole "Pen Tablet" craze of the early 1990's that died. I am reading the original version and should look at the revised version afterwords. The first startup I went to work for, Spyglass, bought copies for everyone to see how we would grow in the first Internet boom of 1995-1996. We ended up trying to be a Gorilla and ended up being kicked to Monkey status by our primary customer Microsoft :).

Wednesday 2011-09-07

Forgot today was Wednesday.. but my wife didn't. She snuck the human-sized cat carrier outside of the bathroom and got me locked into it before I could think what was up. Off to the doctor for me. No wonder I wasn't allowed to get anything to eat after 5pm last night. Visit wasn't too bad, usual prodding and I got a treat for not biting the doctor.

Tasks

• Lots of email from lists
• Spend some time setting up Salt
• I'm not a real DBA, but I get to play one on Fedora Noc.

Commands I want to remember

pg_dumpall > /var/tmp/spew1
mysqldump -u username -ppassword --databases db1 db2 db3 > /var/tmp/spew2
mysql < /tmp/spew2
psql -f spew1 postgres

And xz takes forever... Music

• Kate Bush - Lionheart
• Pink Floyd - The Wall
• The Police - Regatta de Blanc
• Rush - Permanent Waves
• Heart - Bebe Le Strange
• Blue Oyster Cult - Cultosaurus Erectus

Setting Up Salt

Salt is a remote execution manager which allows a system administrator to run multiple commands across many systems in their enterprise at once. Similar to Func, I am looking at it as a configuration tool written in python that I can grok better than I am at puppet.

To start with we worked from the central home server by installing the following rpms:

1. https://github.com/downloads/thatch45/salt/salt-0.8.9-1.el5.noarch.rpm
2. https://github.com/downloads/thatch45/salt/python-zmq-2.1.7-1.el6.x86_64.rpm
3. https://github.com/downloads/thatch45/salt/zeromq-2.1.7-1.el6.x86_64.rpm

$ sudo yum localinstall salt*rpm python-zmq*rpm zeromq*rpm

Now some prep work is needed here. We need to setup the hostname for the salt server in DNS and or /etc/hosts so that our clients can find it. After that we can start the master and minion on our first box.

$ sudo /sbin/chkconfig --add salt-master
$ sudo /sbin/chkconfig --add salt-minion
$ sudo /sbin/service salt-master start
$ sudo /sbin/service salt-minion start

The next clients will only have the minion added. The master will be off as we aren't ready to deal with multi-master setups. So what can we do with salt? Well we can run cmds and such

$ sudo salt '*' cmd.run "ls -l /etc/hosts"        
xanadu.int.smoogespace.com: -rw-r--r--. 1 root root 209 Sep  7 14:59 /etc/hosts
xanadu.int.smoogespace.com: 

$ sudo salt '*' user.info smooge
{'xanadu.int.smoogespace.com': {'gid': 500,
                                'groups': ['desktop_admin_r',
                                           'mock',
                                           'wheel'],
                                'home': '/home/smooge',
                                'name': 'smooge',
                                'passwd': 'x',
                                'shell': '/bin/bash',
                                'uid': 500}}
$ sudo salt '*' status.diskusage /
{'xanadu.int.smoogespace.com': {'/': {'available': 13899972608,
                                      'fstype': 'ext4',
                                      'total': 16910295040}}}

However I then ran into some limitations. I am trying to figure out what might have caused them, but they are rather interesting:

salt '*' pkg.list_pkgs
{'xanadu.int.smoogespace.com': {}}

salt '*' file.get_sum /etc/passwd    
{'xanadu.int.smoogespace.com': 'Hash md5 not supported'}

There should be quite a bit more packages there than that, and I think md5sum is supported. I am guessing there are some packages I need to install or check in the files. Will get to that in my next blog on this.

Tuesday 2011-09-06

Yesterday was Labour Day Holiday in the United States so while I expected to labour through the day, I ended up playing a card game called Dominion with an old college friend. So I guess I spent the day on holiday after all.

Today is now my Monday with a 3 days of unlooked at email.

Tasks

• Email, a lot of email
• RHIT phone meeting
• Rebuild fas01.stg
• Go through security guides to try and write one for CentOS-6

Music

• Rush - Farewell to Kings
• Pink Floyd - Animals
• Meat Loaf - A Bat Out of Hell
• Fleetwood Mac - Rumours
• Heart - Little Queen
• Rush - Hemispheres
• Blue Oyster Cult - Some Enchanted Evening
• Heart - Dog & Butterfly

Friday 2011-09-02

Stuff

Late start today.. the grass pollens are making it very hard to sleep at night. A sudafed with coffee seems to have gotten the day going today though.

Tasks

• Email.. someone got busy with libvirt patches last night
• Spend afternoon trying to build db02.stg

Music

• Pink Floyd - The Division Bell
• Pink Floyd - Meddle
• Fleetwood Mac - Fleetwood Mac
• Pink Floyd - Wish You Were Here
• Rush - Caress of Steel
• Rush - 2112
• Blue Oyster Cult - Agents of Fortune
• Heart - Dreamboat Annie

Thursday 2011-09-01

Well today is turning into one of those "why did I get out of bed" days. I could whinge on about it for a while but what does that fix? Trying to figure out the new blogger interface which just seems odd at the moment.

I have to find a new virtual system provider. My colocated system died a horrible death last night and is down to 1/2 of a RAID and 1/2 of a DIMM. The fact that it has survived 6 years in a closet in my old bosses house in Los Alamos is a wonder.  

Tasks

1. Lots of going back and forth between home and school today.
2. Not as much email since I whacked a bunch accidently
3. Saw how much disk space is in fedora-archive and how long it can take to do a sha256 on it
4. Begin list of Compass goals for the year... now what did I get done again...

Music
Not a lot listened to as I either had to answer silly phone calls from people trying to sell me stuff OR I was going back and forth to the elementary school.

1. Pink Floyd - Dark Side of the Moon
2. Pink Floyd - The Division Bell

Books

1. Geoffrey Moore - Crossing the Chasm