References and Bibliography research

[[UPDATE, October 16, 2026: 

 

I have not used the methods mentioned here for a long time.  I have used JabRef to sort and weed a bibtex file, but the better way is always the simplest way, and all of these fancy schmantzy tools make things way more complicated than is comfortable for me to deal with.  I use these Free Software tools today.  I owe a massive debt of gratitude to the developers of these tools.  

1. Emacs bibtex mode (of course)
2. Cb2bib, a modified version mentioned elsewhere in this blog.  The modification enables me to use a different field (annotate) and, by modifying the "bib2pdf" script included, print an annotated bibliography with added headers and footers (date, time, filename, etc.)   This is a VERY sophisticated tool that I do not fully understand; but I have, as usual, adapted it to my (admittedly obtuse) workflow.   
3. Sioyek.  I think the GNU/Linux version is a little bit crippled, which is sad, because this is a VERY sophisticated program for reading pdfs, and notetaking.  Maybe  I should spend more time learning to use it well, but I understand that some functionality is not well supported on GNU/Linux.  
4. Also, yazi is a hell of a tool for scanning directories, and previewing files.  It is my goto image previewer for sorting through files of esp. jpegs.  Admittedly, it is difficult to configure it for enlarged views of graphics files.  I use it with the excellent shell kitty.   

]] 

 

I am still toying with various research tools. Today, I compiled a little list of references for the marine midges Pontomyia spp. Using JabRef I was able to compile a well-formated bibliography in about an hour and a half, with over 40 references. I think it will be faster once I figure out how to do it all automatically. This was clearly a victory, however.

Google Scholar offers the preference of using bibtex citation export mode. A friend mentioned he's been struggling with bibliography formats. Perhaps a little introduction?

BibTeX is a database format for bibliographic citation for use with the TeX and LaTeX typesetting tools. TeX is a fullblown typesetting language based on Donald Knuth's extensive study of "meat space" typesetting techniques of real printers/typesettings. LaTeX is the macro package, easier to use than TeX. I use LaTeX. These systems are remarkable for hundreds of reasons that I cannot go into at this point. Among them is the handling of references, cross-references, and bibliographic citations. BibTeX is the method for the database. A typical citation is as follows:

@ARTICLE{ruxton2008cea,
author = {Ruxton, G.D. and Humphries, S.},
title = {{Can ecological and evolutionary arguments solve the riddle of the
missing marine insects?}},
journal = {Marine Ecology},
year = {2008},
volume = {29},
pages = {72--75},
number = {1},
publisher = {Blackwell Synergy}
}

I found a great JabRef improvement. Two actually. One of them is Jab2HTML, that enables an export handler that outputs excessively beautiful html from a bibtex file or the entries one has selected. The big thing is the html has links to googlescholar. Secondly, this guy has now put together the means to incorporate the googlescholar links on the summary view of jabref itself, without outputting html:

- http://keijisaito.info/arc/biblio/en_jabref_bibtex.htm

His explanations are for Windoze, but they work ok if you change to Unix directory format, and define your own directories when that comes up. I installed both Jab2HTML and the Pref-* file. I like the Pref- changes alot.

Bibliography Formats


I do almost all text output formated via LaTeX markup. I have found emacs org-mode very, very useful in a number of ways. Org-mode outlines are eventually exportable as LaTeX source, and it works well. If I do this in org mode--- "/Pontomyia pacifica/"---the LaTeX output will be italic, and *xxx* would output xxx in bold. _hello_ would render hello as underlined. This and other matters, including the easy table editor, are really useful. I admit I've spent way too much time learning how to use it, but it's starting to make sense. Todo lists, notes, whatever, preliminary notes. I should be typing this letter in org-mode, one good reason to use Gnus for email (it's way too hard for reading mail with many tags, but easy to compose) and with the emacs-based w3m browser, I can cut and paste text between email and www and whatever I'm writing. I can compose long email messages in org-mode. I understand some people blog from org-mode. You aren't going to want to know about this, I guess, but another reason for me to stay focused on Emacs.

BibTeX gives you a bibliographic database. Emacs is a good way to edit bibtex (*.bib) files, but JabRef and kbibtex and cb2Bib all are essentials to the way I work so far. Zotero I did use today, but I'm not up to speed with bibliography use. I am using it for making notes, like google notes on steroids, but then I can have automatically generated bibtex entries for the source pages for these notes.

I just put together a bibliography in an hour and a half this morning on Pontomyia spp. marine midges. I'll try to attach the html output. What I think is amazing is the speed and relatively clean output. Note the links to google and google scholar.

When one is writing a report---say I am writing a report or proposal for working on Pontomyia. I refer to the *bib file. I'll try to attach that too. Boring.

Two years ago, one wouldn't have found an open specification file format used so widely, for example with Google Scholar using it to export bibliography citations.

I can refer to pontomyia.bib in my proposal and then cite a particular article by Cheng\cite{cheng1978mmp}. If I am publishing an article in Science, I invoke the science citation style, and both the citation ([1] for example) and the formatting of the text in the bibliography will be taken care of by LaTeX/BibTeX. I know it's possible to output a fully formated annotated bibliography by including annotations or notes fields in the bibtex database entries. LaTeX/BibTeX will only print those references in the database that are cited in the article, and one can do all kinds of specific formatting, like including page numbers in teh citation \cite[39-40]{cheng1978mpp}.

LaTeX, by the way will automatically do a crosseferenced bibliography, for example, and correct the page numbers each time the document is processed. Indexing and glossaries the same.

More Notes and References Software

These are somewhat heavyweight, and for the most part are beyond my modest needs and ability. TiddlyWiki NeuroScholar (Demo) RRiki RefBase

 

Relying on a 12 year old recommendation, I will add PDF flip book creator.   Thanks.

Update on cb2Bib

On Ubuntu Hardy, I had to install the bibutils package by hand to get better functionality of cb2Bib: try this link. Bibutils compiled nicely, but didn't make a deb package. I had to install tcsh to run ./configure.

Bibutils is found here

Bibliography tools, continued.

.

cb2Bib

This is a very interesting package. It means "clipboard to bib." Does make picking up references much easier off of web pages. It apparently can do things automatically if one teaches it how, at least some things.

As it is, it right away picks up what it can, like journal name, pages, and volume, then I have had to select the title, or authors into the clipboard, and it asks me what's the category. Very nice.

Zotero

Got it working with Firefox 3 b5 on Ubuntu. Very nice. I'm still a bit puzzled. I was able to save some snapshots and links, and use cb2Bib to massage them into bibtex database entries.

How to get the pdfs on my system integrated, and downloading pdfs? I know kbibtex and JabRef can do some of this.

Zotero w/ Firefox Beta 5

How to get Zotero working w/ Firefox Beta 5 (Ubuntu Hardy Heron)

I've been wanting to test Zotero for more serious research. FInally I have it working. Here are the steps, pretty easy: (28 April 2008)

• Download THIS FILE. (right click, yada yada)

  
  

• Open that file with Firefox (File > Open File)

  
  

• Restart Firefox
  

I can now add tags. Zotero looks more polished.

Ubuntu Essentials

In this space I will outline several essential tweaks, installs, and configurations that FOR ME have improved my experience with Ubuntu. I am writing this on the day that Hardy Heron has been released. My productivity has improved with Ubuntu. I don't have to spend so much time tweaking and configuring, as I did with Gentoo. [Gentoo is still probably better, but some things I never could get to work w/ Gentoo work perfectly with Ubuntu out of the box].

Rules

Rule # 1. Learn to use the VC, which means virtual terminal, and the command line. For a plunge off the deep end do "Ctrl-ALt-F1" simultaneously.

Rule # 2. Set a root password.

Rule # 3. Make a separate /home partition if you use your machine for valuable work (writing, programming, correspondence, artwork, you name it). You WILL have to reinstall the system at some point---in days, or in years. Avoid being a victim of the "it's time to reformat your hard drive" people at the local windows shop.

Rule # 4. Immediately set your repos. System -> Administration -> Software Sources.

Rule # 5. Don't believe all the rules you hear.

Issues with Ubuntu

I am crazy about Ubuntu, but I do have some issues. For starters I will do a "mind sweep" off the top of my head:

Emacs is not installed by default.

Tracker, the search tool, is a pig: it took me several days to figure out where my CPU cycles were going, and how to turn Tracker off (the Daemon and the Client).

Slocate is crippled out of the gate, and needs to be opened up.

/etc/fstab in the new Ubuntu is a thorny knot with hidden pitfalls for the unwary, and, yes, for the wary as well.

Even though I have taken pains that VLC would be the default DVD player and *avi player, etc., the gnome default keeps popping up.

Gnome does not allow me to tweak enough. Some of the nags I have cannot be solved (at least easily).

The installer makes it all too easy to make partitioning mistakes. Most importantly, it is too easy to overwrite a /home partition, and too easy to wrongly partition a favorite directory. Another issue: I have often had to start up sfdisk in a partition to recall which partition I wanted to call / or /home.

About /etc/fstab again---there is a serious bug that shuffles the identities of paritions. It moved an ide drive into /dev/sda and the former /dev/sda (for years) became /dev/sdb . Major bork. It is so confusing, I am never sure when /dev/sda means what. I am very afraid.

It's pretty hard to get to the point of compiling packages. This is a long standing Debian issue with me.

XEphem is not installed by default.

I'm a bit intimidated by Ubuntu's online docs. Gentoos docs are thousands of times better. Well, with Ubuntu you can find what you want, but it's more confusing, and you will find some advice that is best not followed, from well-meaning users who don't know better. Keep this in mind. That being said, I have found some really excellent HOWTOs on line for Ubuntu. Skepticism is well rewarded.

Coolness of Ubuntu

Gnome is working really well. Feels right.

Wireless access is as simple as pie.

Printers (at least HP) are installed before installing them.

Thunderbird "just works" for Gmail.

Even complicated applications, for example for video editing, are working for me out of the box. This was not the case 2 or 3 years ago. I was afraid of this, but I have been pleasantly surprized.

Important Steps on a new Install

$ sudo su

# passwd

(Add a password for root). Now I can run su in a terminal when I want to. This works for me. I am actually becoming more friendly with sudo anymore, though.

Get rid of Tracker.

Edit slocate configuration and enable indexing of more directories.

Install Emacs.

Put a terminal launcher on the panel.

Add repos and update. Can take "forever".

Install a number of programs that I use commonly. In fact, this is where I'm planning to develop a script of other means of my own to get my important utilities installed. It took me a full 24 hours to get the current install up to minimal usefulness, and I had to remember each step, one at a time. Examples: enscript, gthumbs, texlive, ghostview or gv, for starters. Dillo, filelight, vlc, k3b, numerous codecs, amarok, alsamixergui. Some things few would want. These can be taken care of with an edited list made from "dpkg --get-selections"

Turn off unneeded services. (System -> Administration -> Services).

Edit the "Applications" menu and add a submenu for ME. Drop favorites into it.

Add numerous high end Directories to the Places "Bookmarks".

Install dict and many dictionaries.

Install build-essentials, and compile xephem from source.

Recovery problems I have known

I have to admit, the recovery problems have been getting fewer and farther between. That being said, they haven't gone away.

Borked desktop. It always takes me by surprize, the desktop won't work anymore. I have had to delete .gconfd .gconf and .gnome2, perhaps other dot directories, to recover a borked desktop. This is easier if you can log in as root form a VC.

Notable Packages

I am starting to like GNOME Commander. Hope it's not all show and no go.

Nautilus is worth learning to use properly. You can drag a directory onto the sidepanel. This has saved me hours and hours.

Synaptic is better than it used to be. A shopping trip is worth your time.

Things I'd Like to See

Filesystem / Partition icons on the desktop should show device names, not sizes. I tried to fix this, but go lost in /dev/fstab, almost losing the whole install.

Kudos to the Ubuntu team.

Time Enough At Last?

If I'm going to write about literature, libraries, bibliographies, it is obligatory to begin with some specific mentions.

• Time Again At Last, episode8, season 1 of the Twilight Zone, aired on 20 November 1959. Always remembered, this dramatic depiction courses through the streambeds of my life in a particularly haunting manner. A man who wants to do nothing but read, at last given all the time in the world, in the end is deprived, by the shattering of his glasses. Who could not forgive you for downloading this in any manner possible?
• Johann Gotthelf Fischer von Waldheim. (Link to article in German) Steven Jay Gould wrote a piece about him. I named my third son, Thomas Gotthelf Fisher after him. In brief, as Steven J. Gould relates in one of his columns in Natural History, he built up a medical school at Moscow University, the entire library---and every one of his personal books except one---were burned when Napoleon entered Moscow. His appeal to scientists around the world netted a 20,000 volume library in, if I recall correctly, a year or two. In my isolation in Chuuk Lagoon I was brought to tears by the story. What is my personal interest in this story? Briefly, almost all of my books, scientific specimens, instruments, and whatsoever, were lost in Chuuk, to Typhoon and Fire. Most of it will never be replaced. Who cares?
  
• Libraries have been a personal bugaboo of mine, for as long as I can remember. When I was a child, I remember almost always being late to return library books. I never received my University diploma, partly because over $1,500.00 in libary fines stood over my head. I had, you see, two faculty library cards in my wallet and I had been lax in returning the books I had checked out as a work study library researcher for these two professors.
• I have recently been attempting to recover the literature for my MS Thesis research, begun in 1984 (24 years ago!). Most of it was lost in Chuuk. Who knows where it all went. I have been living in a purgatory of sorts, a well trained and competent library researcher, with no access to books.
  

In 2008, one can happily report, it is becoming easier and easier to access literature on the Internet. There are problems, however. The yield per unit of effort is abysmal, even still. And despite much good literature now being available free online, as pdf files, most of it is not. One's interests are necessarily forced by what is available to him.

Yet, in an endeavor such as mine---retrieval of specific important historical papers---the gates are all too often closed. Jstor tries to own the world---can't we attempt to bring a class action suit against Jstor when they attempt to charge 32.00 or maybe more for a paper that the publisher has available free on line? And to charge 52.00 for a paper written by a scientist who has been dead for 100 years? Gads!

So I am gathering together the references, getting what I can off the Inet, and trying to find other ways, perhaps through a friend. If I am fortunate enough to visit the North American continent this summer, perhaps I'll be able to do some diligent literature searches.

For now, I have started experimenting with various ways of handling references for bibtex, the bibliography component of LaTeX, the markup/typesetting macro language I am using for virtually all of my writing. I will forego the discussion of LaTeX. See the TeX User's Group site.

I have been experimenting with the following (not in order):

1. a number of utility programs for manipulating bibtex citation data bases.
   
2. gbib
3. kbib
4. kbibtex
   
5. Jabref
6. Referencer
7. Emacs bibtex mode
8. Pybliographer
9. cb2Bib

I have been amazed at the outcome of these experiments; but as yet I have not been able to get my head around the best of them. They do more than I could have ever imagined, and I'm not sure how.

In future posts I will review mainly for the following:

1. kbibtex,
2. JabRef
   
3. cb2Bib, and
4. Emacs BibTeX mode.

Referencer is also interesting, but the file format is not as easily compatible. I will continue to experiment with it, but as it stands, it is not a serious contender.

Followup on gscan2pdf

I have spent quite a bit of time over the past 24 hours working with gscan2pdf. One document was unworkable, and another worked out with an estimated 95+% accuracy. I was attempting to ocr the document. The ability to save a pdf is a bonus. I could have typed all the pages in over the time it took me to get through this. As it is, I will be able to clean it up in minutes. See the note at the end of this post about Ubuntu.

Document 1: 0% OCR success.
An ancient printout on a 9 pin dot matrix printer, faint due to worn out ribbon. I attempted to deal with various settings for scanning (not many settings possible from the interface of gscan2pdf), unpaper (the options of which I understood but little, if at all), and the OCR---I specified tesseract.

0% isn't good. I will attempt to use the methods described on line using a tiff file and some tweaks. Much too much work.


Document 2: a dark, inkjet printed copy, about 9 pages, with hand written edits on the page.


Discussion and Results:
After spending some hours working with Document 1, with NO effect observed, I was pleased that Document 2 ran through gscan2pdf with almost perfect OCRs. I was displeased that I had to select and paste into a file using an editor. Did I miss something?

I did nothing to the optoins this time around: mostly defaults, except setting the language to English, and setting the scan dpi at 500. Fewer would perhaps work. It didn't take too long, though.

This is more work than one would like to have to do to get editable copies of a stack of pages. Not bad at all, and the next time around, I won't even try with faint dot matrix copies.

NOTE ABOUT UBUNTU:

This is another instance where Ubuntu has it right, or at least right enough to make my life easier. Ubuntu does share the Debian concept regarding compiling kernels and packages that gives me fits and starts once in a while. Productivity is improved at least over the short haul. I need to reflect on this a bit.