Notes on Free Software Usage

Noteworthy Emacs command: dired-do-async-shell-command

2009-06-08T02:05:00.000-07:00

Keybinding: "&".

Just what I needed. I needed to open a *.dvi file from dired, and open the LaTeX source file while looking at the formatted text.

Perfect!

2009-03-14T17:52:00.000-07:00

Unit Conversion

Neither Google nor ConvertAll know about Scruples. Both are convenient ways to convert units. The unix console utility (non-GUI) Units is my choice, so far, as it knows at least one unit, about which the two flashy tools do not.

I installed ConvertAll through Synaptic on Ubuntu. It is written in python using the QT library, so it is usable on Windoze as well as GNU/Linux systems. It's a GUI solution. It doesn't know about scruples, but it is claimed to work well with composite units, such as Kg/m/hr

In the google search bar, I enter "2 scruples in g". Google doesn't know about scruples either.

Units wins the first round. And it's easy to use.

Some ideas for new projects

2009-03-07T21:42:00.000-08:00

Weekends are good. This morning I started poking around the Inet. Here are a couple of nice things I found out about that I can use in my classes. That NFWF grant we lost because we had so much trouble spending out money? That would come in real handy just now.

GeoWall?

Today, I discovered this page: A tour of the Mariana Subduction System . Guess what? It's available as an "ArcScene GeoWall visualization of Mariana 3-D topography and earthquake distribution".

It can be done in GNU/Linux. I think Gentoo would be more appropriate, but one possible option is available out of the box, for Ubuntu, my current system of default choice. One package is "viewer" and it might be around. Another is "inventor". It looks hairy. And why not? It involves projecting a scene through two projectors, superimposed. Let me guess: with polarized glasses.

I saw such a system at the University of Texas Texas Museum of Natural History---a flythrough of a vertebrate skeleton. Which vertebrate? Don't want to spoil it for you.

Bilko

http://www.noc.soton.ac.uk/bilko/index.php

...for learning and teaching remote sensing image analysis skills. Current lessons teach the application of remote sensing to oceanography and coastal management, but Bilko routines may be applied to the analysis of any image in an appropriate format, and include a wide range of standard image processing functions.

Supported by UNESCO, Bilko is available to registered users absolutely free!

Machine meltdown experience, unnumbered

2009-03-04T00:47:00.000-08:00

This week I've been working out how to synchronize systems using git. I've been monkeying around with Emacs Org-mode for a while, both at home and at work, and at infrequent times on a laptop. How can I keep both work environments pretty well evenly matched, so I can work on all of my little projects on both machines (or all three)? Git and a USB flash drive. I won't get into that now, but it's pretty much working out. I have found I need to set the UID (User ID) and/or the GID (Group ID) numbers the same on both systems, and I have implemented a "git" group to which my user (me) on both machines belongs, so there is no conflict when pushing today's work to the repository on the USB stick. It seems to be almost there, as far as my simple use case is concerned.

However, at one point, last friday, I had been working fairly hard on some files that I wanted to be synchronized at home, and I tried to push the changes to the USB stick. The write failed due to lack of permissions, so I sloppily did a "sudo chmod -R a+rw " and was able to recover. After I figured out the permissions might be at least part of the problem! It worked, but in the meantime, I, knowing-just-enough-to-be-dangerous, flailed at the keyboard (I recommend Keytronic), and managed, unknowingly, to mess the settings up in the /etc/group file: on Monday, I was unable to access the groups and users tool to try to solve my problem with permissions---on which I had researched all weekend.

This all to tell you that I was left on Monday with a machine that promised to be unrecoverable. At least by me. But, lo! and behold! On that machine was an unused partition---a Windows partition, and hence an unwanted one. An unneeded one, especially since I was now able to run windows in virtual box as needed. So I quickly installed Ubuntu Intrepid Ibex over lunch, and went back to it before I went home.

Chapter II. Why you need to run dpkg --get-selections

I ran dpkg --get-selections > selections-.

This gives me a list of all the packages on my current system. Lucky my system was at least running! I tucked these away in a file on the current ~/ (home directory), and rebooted the newly made Intrepid Ibex install.

I have installed various GNU/Linuces, dozens of times. Ubuntu is now a favorite because it is painless, and good. It does almost everything. (I still want to get another Gentoo Box going, but the amount of work would be intensive.) My experience has taught me a few routine steps to take to get the machine into working order, MY WAY. It is taking less and less all the time.

This time, however, it went even faster. With my "selections-" file, I started Synaptic, and told it to read this list. (Read Marks). Now the new system has a long list of most of the software that I have working on the old system. It starts downloading them. It says 2d and some. But in the evening the net is faster, and by morning, when I got to work, my machine was up and running, with most of the important bits and pieces I had installed, and also updated for the 4 or 5 months since that CD was assembled by Ubuntu. Pretty impressive. A few tweaks here. A few tweaks there. And the machine is at least functional! It will take little time to be back in the saddle, at 100%, on that steed.

What are my little tweaks? Here are a few

Display some icons on the panel, and place two more panels on the left and right sides. These are made moveable, and expandable, and put at the top, extending down from the top panel, so extra icons can be spilled over onto them like ears. This has been a boon in real estate for icons.
System->Preferences->Windows: set so mouseover raises windows, and to roll up on click of the title bar.
Install emacs-snapshot and a bunch of it's friends.
Firefox add-ons I have met: downthemall, video download helper, tableclipboard tool, zotero, recently flash got, multiple tab handler (allows duplicate tabs, etc), delicious (for the past two weeks, experimental.
On the Firefox bookmark bar, in install a folder called "WLOG" (weblog) where I can pull the little icons from the left end of the location bar to make a bookmark. Just pull it over and drop it onto the WLOG folder. Very useful.
Sometimes compiz configuration tool (forgot the name). The main thing I miss on Gnome is a way to move windows from workspace to workspace by dragging. Spinning cube is nice, but too much of a drag.
Set up an array of "folders" (directories). I haven't figured this one out---perpetually seeking a simplified, hierarchical arrangement that isn't difficult to navigate.

FOLDERS:

I establish too many at the level of ~/: Library (link to separate partition for PDFs); Downloads (where I direct all downloads---this is a candidate for modification, as I would prefer to sort things now rather than later), Work related main directory.
I've been keeping old work around for longer than I am willing to remember. So many subdirs from projects of years ago are still hanging around---all of them meaning to be worked on sooner or later (maybe org mode can help with this). I have the following top level directory trees set aside for various kinds of these: Projects; WORKBENCH; STUDY.
New idea: ~/SCRAPBOOK for all the stuff that hangs around the Desktop and ~/ directories that just stays around and I am unwilling to get rid of. Maybe ~/ATTIC is next.
Another idea that hasn't found it's correct niche: ~/TOP_SHELF. I thought of this after I tried a little applet called top shelf, a place to store things short term, one means to get back to right away. One of the nice things about the new KDE is the window-like desktop. I have tried to implement something of this kind, with no borders, just a place that can display---or be temporarily out of the way. There seem to be two kinds of these top shelves: the top shelf that is too high to reach, with stuff one wants close around; and the back shelf just above the desk where a few cherished and often thought about and used items reside. Need to rename these.
~/MUSIC; ~/VIDEOS; ~/IMAGES (for teaching stuff); ~/PHOTOS.

I also install several programs by hand, so I immediately install build_essential. The main programs I do by hand are XTide and Xephem. Each of them requires other libraries and tools, so I install them.
Gscan2pdf is getting good. My scanner is problematical. When it is working, gscan2pdf is working too.

Chapter III. Permissions and Flash Drives

That's another story.

Beauty more than skin deep: Guitare, etc.

2009-01-17T19:03:00.000-08:00

One of the reasons I refuse to go back to Windoze is my experience with a few special programs---programs that were written by programmers to "scratch an itch," and made available for "anonymous" ftp or download, somewhere "out there," for use by others should the need or wish arise. This restores or sustains my beliefs concerning the nature of man, and sustains my hope and happiness in the present. I cannot imagine why windoze "weenies" don't get this. How misguided is that scene.... One can only admire the prowess of others who have that ability; but perhaps in some other ways I can make my prowess (such as it may be) available to others. Reminds me of the questions people used to ask about the "learning curve" of GNU/Linux, or of LaTeX, or Emacs: the bottom line was, though, that for one willing or able to put in the time to learn, the rainbow gradually became visible. Many of us never went back. Many of the others don't understand why that may be: a friend pointed out to me the other day that Windoze no longer crashes dozens of times a day, as it did when I bailed and started using Linux at the command line, Pre-Linux 1.0. He still missed the point.

This weekend I have had reason to get back in touch with an old friend---the program "guitare."
guitare had installed several years before on one of my GNU/Linux boxes as a command line program, and proved extremely interesting, with capabilities unmatched by any other program I have seen (especially the two in bold face below):

name a chord when the fret numbers of all strings are input
alter the tuning, and do the same in any arbitrary tuning
serve as a chord chart
do much the same for scales

Two years ago, I had sent an email message to Pascal Obry, the developer of the amazing ADA program from yesteryear, "guitare" and the GUI version "tkguitare", explaining how I had not been able to install the program on my more recent machines. I had given up and let it all slide, I don't remember why. Perhaps I had final exams to write, or got involved in another project. I don't remember.

Recently, I have been fiddling around with the most amazing guitar, a gift from my long lost cousin, Mike for my 60th Birthday: a Little Martin. What a tone, and with precision I have never known in dozens of guitars I have played in my lifetime. What an eye opener (ear opener)! I randomly retuned the strings the other week, in preparation for a restringing: I often do this as I usually do not like to work the strings too much, for fear of deadening them. (Although I now realize that not washing my hands must have an even more devastating effect).

On my new random tuning, that my friend Ken says is "drop D" tuning, I have improvised some chords and songs. I don't want to lose them, as I usually do, because the notebooks have a habit of getting lost or forgotten.

Guitare immediately came to mind. It's terrific to be able to tune the program to my unknown tuning, and explore the chords: guitare tells me the canonical name of each chord.

How amazing the exponential development of the Internet as a builder of capacity, as a library, as a source: I did a gmail search and found the old email of Pascal, and contacted him, asking is I could at least find a way to install guitare as a command line utility. Nothing else I have found even comes close! How bizaare is that? Many GUI guitar-oriented programs have appeared recently, in keeping with the movement towards a more user-friendly GNU/Linux desktop; but none of these programs even come close to Pascal's languising program.

I sent email to Pascal:

I'm not sure how to proceed, but what about a version with console only access?

That program is FANTASTIC. I play in all kinds of odd tunings, and using your program I can take my tablatures and find the chord names, adapted to my tuning. And I don't really need tcl or tk, maybe, for that. I don't know of anything else that comes close. (Sometimes the word "beautiful" doesn't refer to superficial appearances, but to a good heart: this program has a good heart!)

Within 24 hours of contacting Pascal about my inability to install his program on my new Ubuntu boxes, he emailed me that he has prepared a new version, v.8, now already available on the guitar web page, as source, GNU/Linux executables, and Windows XP executables, for both guitare and tkguitare. He calls this "Postcardware" and asked that, if I can get it working, to send a postcard.

Installing them on a 32 bit laptop was immediately successful. I had to install the following packages: tcl tcl8.4 tcl-dev tcl8.4-dev tk tk8.4 tk-dev tk8.4-dev . To be sure, maybe I didn't need to install them all, but I had done so in trying to compile earlier.

Tkguitare worked flawlessly. As I wrote to Pascal:

You have won your postcard. But one little thing is still bugging me. The program tkguitare ran fantastically, and dazzled, on a 32 bit Ubuntu laptop--perfect for me, because it's easy to set up when I'm playing the guitar. However, on my 64 bit magnum box, also Intrepid Ibex, like the laptop, it complains with the following:

teitoi@hardware:~$ tkguitare
tkguitare: error while loading shared libraries: libtcl8.4.so.0: cannot open shared object file: No such file or directory
teitoi@hardware:~$

One suspects that this is because I'm on a 64 bit box? I made double sure that all libraries (tk, tcl, tk-dev, tcl-dev) were installed. guitare, console mode, works fine. What a relief!

Still, I have to say, in this case you have proven that a thing that has a beautiful heart can also be beautiful in appearance. And that beauty adds a good deal of functionality and useability.

So I still had a problem on the 64 bit box. After sending Pascal an email asking whether he knows a fix, I continued to try several options: I tried to compile, but failed again; I looked all over the repos for 32 bit libraries for tcl. Finally, I found a message mentioning a program called "getlibs" that finds libraries for 32 bit binaries on a 64 bit Ubuntu system. Having nothing to lose, I installed it, after a google search. Here's a link to the Ubuntu Forums discussion:

http://ubuntuforums.org/showthread.php?t=474790

Within 5 minutes tkguitare was working on my 64 bit box. WOW!!!!!!!!!!!!!!!!!

By the way, Tab-n-Fret is a package that long ago enabled me to link a graphical GUI of a fretboard (xfretboard) to emacs, and use it or a non-gui method to input tabulature in text mode in Emacs. Perhaps this will be my next project. For the nonce, however, another program, etktab, is available on the Ubuntu repos, and is built from the Tab-n-fret, and tabulature mode and xfretboard from that package.

Org-Mode and VLC

2009-01-12T16:39:00.000-08:00

Vlc is a wonderful media player. For movies, it automatically integrates a subtitle file if one is available by the same base name as the movie file.

Org-Mode is a wonderful outlining, notetaking, organizing utility for Emacs, the editor I use to do almost everything involving text of any nature.

Carsten Dominic has written a Relative Timer feature into Org-Mode. I use audio-visual materials quite alot in teaching, and I have often taken notes of videos or movies, to facilitate development of a lesson or a worksheet. The Org-Mode relative timer is a fantastic help, and Carsten has now made it better, integrating a facility to pause the timer.

I discovered an emacs lisp routine "dired-mplayer" at http://www.emacswiki.org/emacs/EMMS , and adapted it very crudely to start up vlc from emacs dired. It now starts vlc asynchronously, so emacs doesn't freeze up waiting for the video to finish, which leaves me free to take notes using Org-Mode and the Relative Timer feature. But pausing is what I needed, and Carsten's new feature makes that work.

I have incorporated a facility in dired-vlc that will simultaneously open a note buffer in org-mode in Emacs, and start the timer.

At this point, I use an org-remember template to open a buffer in a file of video notes. Then, insert a time stamp at t=0, and proceed.

I assigned the command to pause and restart to F11, so I can pretty easily pause the video and immediately afterwards pause the org-mode timer. Same for restart.

To make this more useful, I hope to incorporate code to pause the video and the timer simultaneously, wait for a note, and then restart both on command.

Here's the code for dired-vlc. It's more than just rough, but it works.

;;--------------------------------------------------begin dired-vlc.el---------------------
;; -*- mode: elisp -*-
;; Time-stamp: <2009-01-13>
;; AED 05 January 2009
;;;;;;;;;;;;;;;;;

(require 'org)

(defvar dired-vlc-program "/usr/bin/vlc")

(defun dired-vlc (&optional timer)
"Asynchronously start vlc on file through dired. If an optional
argument is given (C-u), the org relative timer is started. This
function purports to start vlc in rc mode, to leave open the
possibility of remote control."
(interactive "P")
(let ((file (expand-file-name (dired-get-filename)))
ext files basename dir curr-file ;idx-file sub-file srt-file
command options)
(setq basename (file-name-nondirectory
(file-name-sans-extension file)))
(setq dir (file-name-directory file))
(setq files (directory-files dir t basename))
(delete file files)
(setq command (format "\"%s\" \"%s" dired-vlc-program "--intf rc"))
; file))
(if (y-or-n-p (format "Run command %s?" command))
; (shell-command command))))
(start-process "junk" nil dired-vlc-program file)))
(if (equal timer '(4)) (org-timer-start))
)

;; end dired-vlc.el

2008-12-07T05:46:00.000-08:00

Distribution Alternation

I have moved my machines to Ubuntu. Beginning immediately on the day of its release, I also moved all of them to Intrepid Ibex, Ubuntu 2008.10. Tonight I want to report that I am pleased after a somewhat lengthy trial period: I will stick with Ubuntu for now.

I have looked around for a replacement, and very few catch my eye anymore. This is a major change for me, as I have been an inveterate tester of distros. From Slackware to Debian to Knoppix to Mepis to Gentoo to Ubuntu to Gentoo again, then to Ubuntu. I think this captures the main thrust of my experiments, albeit there have been some more or less casual and brief detours along the way---for example when I installed cluster knoppix on three machines in my classroom, upon a time.

I had tried Gentoo for two or three main reasons: to challenge my fear of confiuration; to find a distro to run both X86 and AMD64 executables on (I'd read that Gentoo did this best); and to search for more solid performance. Gentoo was successful in all of these areas, and more. Not only that, but Gentoo had the best documentation in the GNU/Linux world, hands down, until recently.

Several problems tugged me away from Gentoo this time. I don't have enough time to maintain three or four systems, let alone one, when it takes me often several hours or longer to track down a problem. That's the fault, perhaps, of my lack of formal Computer Science training. For various reasons I do need a working computer, and a reliable one. Ubuntu has gotten to the point where it works well out of the box, and the hard problems are fewer and farther in between. (That being said, K3b on Gentoo works fantastically out of the box, and verifies burned CDs almost always; K3b on Ubuntu Intrepid Ibex is unable to verify a burn of an iso image. FOr me.

Another reason is the ease of installation of programs. It's plenty easy to install programs on Gentoo, and I like the method better, in fact. However, the advantages do not seem to outweigh the main disadvantage (time taken to install) anymore.

Gentoo seems to be falling apart. I would like to volunteer to help keep it afloat. But I am so troubled by even trying to keep a system working withing reasonable parameters, and up to date, that I cannot expect that I would be able to even offer reasonable assistance with documentation.

In particular, the docs are not as good as they used to be.

References and Bibliography research

2008-05-10T16:43:00.000-07:00

I am still toying with various research tools. Today, I compiled a little list of references for the marine midges Pontomyia spp. Using JabRef I was able to compile a well-formated bibliography in about an hour and a half, with over 40 references. I think it will be faster once I figure out how to do it all automatically. This was clearly a victory, however.

Google Scholar offers the preference of using bibtex citation export mode. A friend mentioned he's been struggling with bibliography formats. Perhaps a little introduction?

BibTeX is a database format for bibliographic citation for use with the TeX and LaTeX typesetting tools. TeX is a fullblown typesetting language based on Donald Knuth's extensive study of "meat space" typesetting techniques of real printers/typesettings. LaTeX is the macro package, easier to use than TeX. I use LaTeX. These systems are remarkable for hundreds of reasons that I cannot go into at this point. Among them is the handling of references, cross-references, and bibliographic citations. BibTeX is the method for the database. A typical citation is as follows:

@ARTICLE{ruxton2008cea,
author = {Ruxton, G.D. and Humphries, S.},
title = {{Can ecological and evolutionary arguments solve the riddle of the
missing marine insects?}},
journal = {Marine Ecology},
year = {2008},
volume = {29},
pages = {72--75},
number = {1},
publisher = {Blackwell Synergy}
}

I found a great JabRef improvement. Two actually. One of them is Jab2HTML, that enables an export handler that outputs excessively beautiful html from a bibtex file or the entries one has selected. The big thing is the html has links to googlescholar. Secondly, this guy has now put together the means to incorporate the googlescholar links on the summary view of jabref itself, without outputting html:

- http://keijisaito.info/arc/biblio/en_jabref_bibtex.htm

His explanations are for Windoze, but they work ok if you change to Unix directory format, and define your own directories when that comes up. I installed both Jab2HTML and the Pref-* file. I like the Pref- changes alot.

Bibliography Formats

I do almost all text output formated via LaTeX markup. I have found emacs org-mode very, very useful in a number of ways. Org-mode outlines are eventually exportable as LaTeX source, and it works well. If I do this in org mode--- "/Pontomyia pacifica/"---the LaTeX output will be italic, and *xxx* would output xxx in bold. _hello_ would render hello as underlined. This and other matters, including the easy table editor, are really useful. I admit I've spent way too much time learning how to use it, but it's starting to make sense. Todo lists, notes, whatever, preliminary notes. I should be typing this letter in org-mode, one good reason to use Gnus for email (it's way too hard for reading mail with many tags, but easy to compose) and with the emacs-based w3m browser, I can cut and paste text between email and www and whatever I'm writing. I can compose long email messages in org-mode. I understand some people blog from org-mode. You aren't going to want to know about this, I guess, but another reason for me to stay focused on Emacs.

BibTeX gives you a bibliographic database. Emacs is a good way to edit bibtex (*.bib) files, but JabRef and kbibtex and cb2Bib all are essentials to the way I work so far. Zotero I did use today, but I'm not up to speed with bibliography use. I am using it for making notes, like google notes on steroids, but then I can have automatically generated bibtex entries for the source pages for these notes.

I just put together a bibliography in an hour and a half this morning on Pontomyia spp. marine midges. I'll try to attach the html output. What I think is amazing is the speed and relatively clean output. Note the links to google and google scholar.

When one is writing a report---say I am writing a report or proposal for working on Pontomyia. I refer to the *bib file. I'll try to attach that too. Boring.

Two years ago, one wouldn't have found an open specification file format used so widely, for example with Google Scholar using it to export bibliography citations.

I can refer to pontomyia.bib in my proposal and then cite a particular article by Cheng\cite{cheng1978mmp}. If I am publishing an article in Science, I invoke the science citation style, and both the citation ([1] for example) and the formatting of the text in the bibliography will be taken care of by LaTeX/BibTeX. I know it's possible to output a fully formated annotated bibliography by including annotations or notes fields in the bibtex database entries. LaTeX/BibTeX will only print those references in the database that are cited in the article, and one can do all kinds of specific formatting, like including page numbers in teh citation \cite[39-40]{cheng1978mpp}.

LaTeX, by the way will automatically do a crosseferenced bibliography, for example, and correct the page numbers each time the document is processed. Indexing and glossaries the same.

More Notes and References Software

2008-05-02T06:23:00.001-07:00

These are somewhat heavyweight, and for the most part are beyond my modest needs and ability.

TiddlyWiki

NeuroScholar
(Demo)

RRiki

RefBase

Update on cb2Bib

2008-04-28T04:55:00.000-07:00

On Ubuntu Hardy, I had to install the bibutils package by hand to get better functionality of cb2Bib: try this link. Bibutils compiled nicely, but didn't make a deb package. I had to install tcsh to run ./configure.

Bibutils is found here

Bibliography tools, continued.

2008-04-27T21:26:00.000-07:00

cb2Bib

This is a very interesting package. It means "clipboard to bib." Does make picking up references much easier off of web pages. It apparently can do things automatically if one teaches it how, at least some things.

As it is, it right away picks up what it can, like journal name, pages, and volume, then I have had to select the title, or authors into the clipboard, and it asks me what's the category. Very nice.

Zotero

Got it working with Firefox 3 b5 on Ubuntu. Very nice. I'm still a bit puzzled. I was able to save some snapshots and links, and use cb2Bib to massage them into bibtex database entries.

How to get the pdfs on my system integrated, and downloading pdfs? I know kbibtex and JabRef can do some of this.

Zotero w/ Firefox Beta 5

2008-04-26T16:18:00.000-07:00

How to get Zotero working w/ Firefox Beta 5 (Ubuntu Hardy Heron)

I've been wanting to test Zotero for more serious research. FInally I have it working. Here are the steps, pretty easy: (28 April 2008)

Download THIS FILE. (right click, yada yada)
Open that file with Firefox (File > Open File)
Restart Firefox

I can now add tags. Zotero looks more polished.

2008-04-25T01:00:00.000-07:00

Ubuntu Essentials

In this space I will outline several essential tweaks, installs, and configurations that FOR ME have improved my experience with Ubuntu. I am writing this on the day that Hardy Heron has been released. My productivity has improved with Ubuntu. I don't have to spend so much time tweaking and configuring, as I did with Gentoo. [Gentoo is still probably better, but some things I never could get to work w/ Gentoo work perfectly with Ubuntu out of the box].

Rules

Rule # 1. Learn to use the VC, which means virtual terminal, and the command line. For a plunge off the deep end do "Ctrl-ALt-F1" simultaneously.

Rule # 2. Set a root password.

Rule # 3. Make a separate /home partition if you use your machine for valuable work (writing, programming, correspondence, artwork, you name it). You WILL have to reinstall the system at some point---in days, or in years. Avoid being a victim of the "it's time to reformat your hard drive" people at the local windows shop.

Rule # 4. Immediately set your repos. System -> Administration -> Software Sources.

Rule # 5. Don't believe all the rules you hear.

Issues with Ubuntu

I am crazy about Ubuntu, but I do have some issues. For starters I will do a "mind sweep" off the top of my head:

Emacs is not installed by default.

Tracker, the search tool, is a pig: it took me several days to figure out where my CPU cycles were going, and how to turn Tracker off (the Daemon and the Client).

Slocate is crippled out of the gate, and needs to be opened up.

/etc/fstab in the new Ubuntu is a thorny knot with hidden pitfalls for the unwary, and, yes, for the wary as well.

Even though I have taken pains that VLC would be the default DVD player and *avi player, etc., the gnome default keeps popping up.

Gnome does not allow me to tweak enough. Some of the nags I have cannot be solved (at least easily).

The installer makes it all too easy to make partitioning mistakes. Most importantly, it is too easy to overwrite a /home partition, and too easy to wrongly partition a favorite directory. Another issue: I have often had to start up sfdisk in a partition to recall which partition I wanted to call / or /home.

About /etc/fstab again---there is a serious bug that shuffles the identities of paritions. It moved an ide drive into /dev/sda and the former /dev/sda (for years) became /dev/sdb . Major bork. It is so confusing, I am never sure when /dev/sda means what. I am very afraid.

It's pretty hard to get to the point of compiling packages. This is a long standing Debian issue with me.

XEphem is not installed by default.

I'm a bit intimidated by Ubuntu's online docs. Gentoos docs are thousands of times better. Well, with Ubuntu you can find what you want, but it's more confusing, and you will find some advice that is best not followed, from well-meaning users who don't know better. Keep this in mind. That being said, I have found some really excellent HOWTOs on line for Ubuntu. Skepticism is well rewarded.

Coolness of Ubuntu

Gnome is working really well. Feels right.

Wireless access is as simple as pie.

Printers (at least HP) are installed before installing them.

Thunderbird "just works" for Gmail.

Even complicated applications, for example for video editing, are working for me out of the box. This was not the case 2 or 3 years ago. I was afraid of this, but I have been pleasantly surprized.

Important Steps on a new Install

$ sudo su

# passwd

(Add a password for root). Now I can run su in a terminal when I want to. This works for me. I am actually becoming more friendly with sudo anymore, though.

Get rid of Tracker.

Edit slocate configuration and enable indexing of more directories.

Install Emacs.

Put a terminal launcher on the panel.

Add repos and update. Can take "forever".

Install a number of programs that I use commonly. In fact, this is where I'm planning to develop a script of other means of my own to get my important utilities installed. It took me a full 24 hours to get the current install up to minimal usefulness, and I had to remember each step, one at a time. Examples: enscript, gthumbs, texlive, ghostview or gv, for starters. Dillo, filelight, vlc, k3b, numerous codecs, amarok, alsamixergui. Some things few would want. These can be taken care of with an edited list made from "dpkg --get-selections"

Turn off unneeded services. (System -> Administration -> Services).

Edit the "Applications" menu and add a submenu for ME. Drop favorites into it.

Add numerous high end Directories to the Places "Bookmarks".

Install dict and many dictionaries.

Install build-essentials, and compile xephem from source.

Recovery problems I have known

I have to admit, the recovery problems have been getting fewer and farther between. That being said, they haven't gone away.

Borked desktop. It always takes me by surprize, the desktop won't work anymore. I have had to delete .gconfd .gconf and .gnome2, perhaps other dot directories, to recover a borked desktop. This is easier if you can log in as root form a VC.

Notable Packages

I am starting to like GNOME Commander. Hope it's not all show and no go.

Nautilus is worth learning to use properly. You can drag a directory onto the sidepanel. This has saved me hours and hours.

Synaptic is better than it used to be. A shopping trip is worth your time.

Things I'd Like to See

Filesystem / Partition icons on the desktop should show device names, not sizes. I tried to fix this, but go lost in /dev/fstab, almost losing the whole install.

Kudos to the Ubuntu team.

Time Enough At Last?

2008-04-19T17:52:00.000-07:00

If I'm going to write about literature, libraries, bibliographies, it is obligatory to begin with some specific mentions.

Time Again At Last, episode8, season 1 of the Twilight Zone, aired on 20 November 1959. Always remembered, this dramatic depiction courses through the streambeds of my life in a particularly haunting manner. A man who wants to do nothing but read, at last given all the time in the world, in the end is deprived, by the shattering of his glasses. Who could not forgive you for downloading this in any manner possible?
Johann Gotthelf Fischer von Waldheim. (Link to article in German) Steven Jay Gould wrote a piece about him. I named my third son, Thomas Gotthelf Fisher after him. In brief, as Steven J. Gould relates in one of his columns in Natural History, he built up a medical school at Moscow University, the entire library---and every one of his personal books except one---were burned when Napoleon entered Moscow. His appeal to scientists around the world netted a 20,000 volume library in, if I recall correctly, a year or two. In my isolation in Chuuk Lagoon I was brought to tears by the story. What is my personal interest in this story? Briefly, almost all of my books, scientific specimens, instruments, and whatsoever, were lost in Chuuk, to Typhoon and Fire. Most of it will never be replaced. Who cares?
Libraries have been a personal bugaboo of mine, for as long as I can remember. When I was a child, I remember almost always being late to return library books. I never received my University diploma, partly because over $1,500.00 in libary fines stood over my head. I had, you see, two faculty library cards in my wallet and I had been lax in returning the books I had checked out as a work study library researcher for these two professors.
I have recently been attempting to recover the literature for my MS Thesis research, begun in 1984 (24 years ago!). Most of it was lost in Chuuk. Who knows where it all went. I have been living in a purgatory of sorts, a well trained and competent library researcher, with no access to books.

In 2008, one can happily report, it is becoming easier and easier to access literature on the Internet. There are problems, however. The yield per unit of effort is abysmal, even still. And despite much good literature now being available free online, as pdf files, most of it is not. One's interests are necessarily forced by what is available to him.

Yet, in an endeavor such as mine---retrieval of specific important historical papers---the gates are all too often closed. Jstor tries to own the world---can't we attempt to bring a class action suit against Jstor when they attempt to charge 32.00 or maybe more for a paper that the publisher has available free on line? And to charge 52.00 for a paper written by a scientist who has been dead for 100 years? Gads!

So I am gathering together the references, getting what I can off the Inet, and trying to find other ways, perhaps through a friend. If I am fortunate enough to visit the North American continent this summer, perhaps I'll be able to do some diligent literature searches.

For now, I have started experimenting with various ways of handling references for bibtex, the bibliography component of LaTeX, the markup/typesetting macro language I am using for virtually all of my writing. I will forego the discussion of LaTeX. See the TeX User's Group site.

I have been experimenting with the following (not in order):

a number of utility programs for manipulating bibtex citation data bases.
gbib
kbib
kbibtex
Jabref
Referencer
Emacs bibtex mode
Pybliographer
cb2Bib

I have been amazed at the outcome of these experiments; but as yet I have not been able to get my head around the best of them. They do more than I could have ever imagined, and I'm not sure how.

In future posts I will review mainly for the following:

kbibtex,
JabRef
cb2Bib, and
Emacs BibTeX mode.

Referencer is also interesting, but the file format is not as easily compatible. I will continue to experiment with it, but as it stands, it is not a serious contender.

Followup on gscan2pdf

2008-03-21T22:55:00.000-07:00

I have spent quite a bit of time over the past 24 hours working with gscan2pdf. One document was unworkable, and another worked out with an estimated 95+% accuracy. I was attempting to ocr the document. The ability to save a pdf is a bonus. I could have typed all the pages in over the time it took me to get through this. As it is, I will be able to clean it up in minutes. See the note at the end of this post about Ubuntu.

Document 1: 0% OCR success.
An ancient printout on a 9 pin dot matrix printer, faint due to worn out ribbon. I attempted to deal with various settings for scanning (not many settings possible from the interface of gscan2pdf), unpaper (the options of which I understood but little, if at all), and the OCR---I specified tesseract.

0% isn't good. I will attempt to use the methods described on line using a tiff file and some tweaks. Much too much work.

Document 2: a dark, inkjet printed copy, about 9 pages, with hand written edits on the page.

Discussion and Results:
After spending some hours working with Document 1, with NO effect observed, I was pleased that Document 2 ran through gscan2pdf with almost perfect OCRs. I was displeased that I had to select and paste into a file using an editor. Did I miss something?

I did nothing to the optoins this time around: mostly defaults, except setting the language to English, and setting the scan dpi at 500. Fewer would perhaps work. It didn't take too long, though.

This is more work than one would like to have to do to get editable copies of a stack of pages. Not bad at all, and the next time around, I won't even try with faint dot matrix copies.

NOTE ABOUT UBUNTU:

This is another instance where Ubuntu has it right, or at least right enough to make my life easier. Ubuntu does share the Debian concept regarding compiling kernels and packages that gives me fits and starts once in a while. Productivity is improved at least over the short haul. I need to reflect on this a bit.

I have entered my Ubuntu period

2008-03-17T13:30:00.000-07:00

I have installed Ubuntu on all of my machines. I installed 7.10, first. Now I have been running hardy heron, 8.04, for a month or so on two machines, my main (home) machine, and my main school machine. A laptop and a newer dual core intel machine at school are running 7.10, and the latter is dual booting with Windoze XP because I need to use Windoze for school official business (a perfect BANE). On the laptop I am running VMWare with windoze XP, for school business.

More later on this. A few remarks are pertinent, however:

Ubuntu is easy. Maintaining several gentoo machines can be a bore and a drudge. My original wants for Gentoo were to learn GNU/Linux better, and to achieve better stability after so long of a time with unstable (in real terms) debian based systems. Ubuntu is a debian based system, and nowadays, it works. The test package was avidemux and other video programs. In the old days, complicated programs were complicated to keep running, complicated with lots of problems. This program and others are running out of the box, with few glitches at all. The constant upgrade knots that I had twith Debian, Knoppix, and actually Ubuntu of old (a year and more ago) are not a serious problem, although I have had to intervene once or twice. And now VMWare: was I dreaming? It installed with a single "apt-get install vmware-server", a few tweaks per a well-written howto, including a simple XP install.

The install was still tricky, in terms of getting the particions right. I have been preserving the /home partition with home directories for years, through Knoppix, Debian, Ubuntu, Gentoo, etc. Ubuntu works ok for this but requires a methodical and careful intervention at install time.

Two Weeks: Three Distributions

2007-09-23T03:20:00.000-07:00

Less than two weeks ago, I got a new laptop from work. In the period since I have installed three different distributions of GNU/Linux (4 if you could two different versions of Ubuntu): Ubuntu Alpha (Gutsy Gibbon, early version); Sabayon, a recent DVD; Ubuntu current stable version; and Gentoo. Gentoo won again, hands down; although, Ubuntu stable version did fine for the less than a day it was on my machine.

A brief summary of the four installs will be pertinent.

Ubuntu unstable was indeed unstable. I couldn't connect to the WPA encrypted wireless network at work, the main reason for abandonment; but in reality, the installation became very, very buggy as I upgraded.

Sabayon was another top contender: a gentoo derived, very high glitz level, cool if it worked. In all fairness, I had trouble with the DVD itself, but the install was a mess, with many unexplainable instabilities. I eventually threw up my hands in frustration, and started downloading a Gentoo 2007.0 Live CD and also a Minimal Install CD.

While I was downloading those, I found a CD of Ubuntu stable, and I installed it. I was able to connect to WPA wireless, and I even considered bailing on the Gentoo install. I knew the Gentoo installation would take time, and I was afraid of the WPA issues (and even now I haven't solved that problem, but I feel a little better about it.) Ubuntu stable was better, but the die had been cast.

I have to say that Ubuntu did give me serious troubles partitioning. I have had that trouble before with Ubuntu, and I do not like the utility that is bundled with the Live CD---the Disk Druid. I have botched several partitioning jobs over the past couple of years, due to, I think, the non-intuitive command set for that GUI utility. It's easy, I suppose, if one has made a decision to install in a particular way, but there are too many things that can go wrong.

And did go wrong in my case: the Windoze partition was blown off. I admit I was relieved, and went ahead and repartitioned for Gentoo, leaving a 5 GM partition for Ubuntu if I decide I need a short term bail out, or some extra storage. (This machine has only 40GB of HDD storage!)

Finally, I tried Gentoo with a Minimal install CD. This is a CD that required a working internet connection, the faster the better. I did have a connection at home, but I gave this up when the Live CD was finally burned.

The Live CD worked well. Better than the last time, when the Gentoo Live CD bombed. This time it went well, although I had to study the installation docs carefully a couple of times. I still believe, as I have for a couple of years, that Gentoo docs are the best of all; but the installation docs are so numerous it's hard to keep them straight. I did find what I needed, although I had to watch carefully along the way.

The Live CD has taken several days to get cherried out. As of tonight, about four or five days out, it's working extremely well.

Why do I stay with Gentoo? It takes days to install, a discouraging prospect, but I'm not about to give it a second go. It's the only distro that is solid enough to not require several installation attempts, but it sure better be.

This year's Gentoo is better still. Tonight I installed the most unstable version of Avidemux, and it works perfectly. That is probably a litmus test. Vlc seems fine, and I have been working on Mplayer. Gentoo has taken everything I've thr0wn at it, and that's alot.

An OCR Sccess Story: tesseract, gscan2pdf, and unpaper.

2007-08-31T21:55:00.000-07:00

Wherein I opine that OCR is ready for prime time on GNU/Linux.

For some years I've been carrying around a manila file of a printout of a digital file for a project of mine from the early 90s, a list of observed seasonal events over a two or three year period on Tol Island, in Chuuk Lagoon, E. Caroline Islands. It seemed to be the litmus test of OCR. I didn't believe it would ever be possible to scan and reduce it back to characters. It was printed on an old HP Portable Deskjet printer.

At the time I was using a Toshiba Satellite laptop. A friend had given me two 10 Watt solar panels, really ancient ones. I had scrounged a 100 AHr deep cycle battery from the local Air Force Civic Action Team (CAT Team) when they had to maintain their batteries on their heavy equipment. Every three weeks or so I would carry the battery into town, and the CAT team would charge it. In between, I had the 10 Watt panel to trickle charge the beast. It kept my computer working for about three weeks! Not bad. And I had built an adaptor for my Zeiss microscope, to run it from 12VDC, with a rheostat dimmer, so it was just as good as power mains. So there was a bit of load on the old Deep Cycle battery after all: I spent many hours a day either on the computer, or with the microscope. (The Stereomicroscope that I'd gotten from Jerry Bakus at USC was illuminated by a kerosene pressure lantern).

I jury rigged the printer to run off of 12VDC also. The beauty of this was that by bypassing the power adaptor plug, and wiring directly in parallel with the batteries, I saved a considerable amount of power, because the printer had a built in sleep mode that only worked when it was running off battery power.

This printer wasn't a great one, and I am sure I harmed the quality by refilling the cartridges. In those days, refilling technology---not to mention inket technology---was in a primitive state, and the refilled cartridge, if it worked at all, would not print anywhere near the same quality of a new one. But, I could not possible afford a new cartridge every month or so, and it was only lucky that I had gotten ahold of a pint of ink, so I made do.

Making do was the by word of Terry Frohm, the technician at the Chuuk branch of the Coral Reef Research Lagoratory, in Neauwo. He was a bit of a bricoleur(See The Savage Mind, by Claude Levi-Strauss), and had cobbled together many of the systems at the lab from scrounged articles: the water tank feed was a prime example. I resolved to pull together a book about "making do" beginning with ideas from Terry.

Back to OCR: tesseract, gscan2pdf, and unpaper.

Last night one of page of the printouts of my seasonality files surfaced on top of the pile of my desk. I had recently played around with tesseract. I had read a couple of recent articles. OCR was beginning to surface. Some articles and links:

An article at LinuxJournal on Tesseract This one works!
A gentoo wiki howto on OCR, including some on Tesseract
A howto for Ubuntu and Tesseract
Ocropus
Tesseract (Google Summer of Code)
And the one that makes it all work together: gscan2pdf and unpaper. Here's an excerpt from the gentoo forum about the installation on Gentoo:

Hi, I updated the ebuild for gscan2pdf in bugzilla. I thought I'd put some information about the app here since this thread is the only result when searching for gscan2pdf on the forums. Also, I think anyone who wants to try gscan2pdf might want to know about the issues with unpaper, which is the topic of this thread.

The ebuild for gscan2pdf-0.9.16 includes support for tesseract-ocr. This means that with gscan2pdf, it's now possible to scan in a document, have OCR run automatically, and then when the scan is exported to pdf the OCR text will be attached as a comment or annotation. This text can be viewed using the Acrobat pdf reader, but, more importantly perhaps, desktop search engines like beagle will index it. So this makes pdfs of scanned paper much easier to find, especially if one has a lot of them. Unlike other FOSS OCR apps, tesseract actually works reasonably well, certainly well-enough for indexing purposes. I don't know of another GUI frontend for tesseract.

Of course, gscan2pdf has a lot of other features, including ADF (automatic document feeder) support, creation of multi-page pdfs, thumbnail previews for easy page reordering, export to tiff and djvu, and so on. It uses libsane, gtk2-perl and PDF-API2. More info here: http://gscan2pdf.sourceforge.net/
I think it's definitely worth a try for anyone who wants to scan books, bills, or other documents (and then be able to find them easily with desktop search).

To get back on topic, gscan2pdf also can make use of unpaper to clean up pages after scanning (it's the only frontend I know of for unpaper, too). The issue is that unpaper.c won't compile without some specific, and perhaps broken, compiler options (-ftree-vectorize, in particular). In fact, I can't get it to compile at all on my system, even when I use the same compiler options as used in preparing the binary that is distributed along with the source code. Has anyone been able to compile an unpaper binary from unpaper.c? If someone wants to try, the tarball with the source file can be found here: http://unpaper.berlios.de/

Also, there doesn't seem to have been any response at all to the bug filed with unpaper upstream last April. I'm not sure of the best way to approach the situation at this point. Because I can't make a binary from the .c file, it doesn't seem like a packaging or ebuild problem to me.

As a workaround, I've been using the precompiled binary for unpaper (I copied it to /usr/bin). I do realize that this is not a very good solution. But unpaper really does improve the look of my scans and it also makes the OCR work even better, so I've just settled on this unsatisfying compromise for the moment.

Anyway, although I suppose that the unpaper issues will keep gscan2pdf out of portage until they are resolved, I'd be interested in hearing any feedback on all of this stuff, including the gscan2pdf ebuild I linked earlier. Thanks.

I don't know how you are going to install gscan2pdf on another system. Unpaper is required for the use of gscan2pdf. And gscan2pdf uses tesseract or gocr. I have both installed, but I don't know whether gocr works. It used to be a PITA.

Here are the links for these two bits:

Gscan2pdf is a gui, it can do the scan, the adjustments using unpaper(another way of doing the adjustments is explained in the Linux Journal article: it worked for me also). One more click and it does the scan.

Multiple columns required me to fire up the Gimp, and save the columns into separate files.

I now have a copy of my page of phenology observations in an editable text file. I hardly had to edit it at all: it was remarkably clean.

Discussion of LaTeX flipbook idea

2007-08-31T21:48:00.000-07:00

On the group comp.text.tex, is found a discussion of how to do a flip book included in a book.

The winning post, in my opinion so far, is a suggestion to look at the documentation for the package fancyhdr.

In the beginning ...

2007-08-04T17:09:00.000-07:00

I am starting a blog of notes about GNU/Linux.

GNU/Linux is an important part of my life, but I am not a developer. This BLOG will be a means of organizing my thoughts about GNU/Linux, it's place in my life, and thinking about what I can do to give something back to the Community.

As a Gentoo User, I noticed a need for user feedback and participation, mentioned recently by Danial Robbins, original architect of Gentoo, in his BLOG,

I also need to keep track of my own meanderings, on the continuing learning journey of using GNU/Linux and Free Software generally.