Wednesday, April 22, 2026

How I got into Emacs (and Free Software)

This is a long tale, not confined to Emacs alone. This is, indeed, a story about why I am a confirmed Free Software advocate. This has been a rocky road. For this chapter, I'll try my best to restrict my discussion to Emacs; but that's impossible. In the late 80s and early 90s, I was living in Chuuk Lagoon, Micronesia. Graduate work at the University of Guam (UOG) marine lab had been interrupted, at the outset of my third year; in Chuuk, my wife was preparing to give birth to our first son.

In my letter of intent for my application to the graduate program at UOG Marine Lab, in 1984, I had written that my first order of business was to learn the marine fauna  of Micronesia, to prepare for future study of traditional marine biological knowledge. A lexicon of marine life would be required. After  two years of classwork, with a working  knowledge of the fauna, in late 1986 I hit the ground running, collecting notes on the names and knowledge of marine animals. 


As  notes piled up, I would obviously need a PC to compile, sort, and analyze the burgeoning information. I had a working knowledge of computers, and had used some of the earliest PCs---IBM XT and IBM AT--- at the Marine Lab. Somehow I acquired a PC, and started messing around with typing in  names, and organizing the data.  I  purchased a copy of SNOBOL4, and began to study it, and SPITBOL, hopeful it would be helpful for string manipulation.   I tried Borland Paradox.   Progress was minimal. 

 

In 1990 and 1991, my life in turmoil I moved to another island, nearby. My Mother came to the rescue, gifting me a Toshiba laptop. I soon  realized it came with no software, not even a word processor.  I made do for a while.  I remember using 4Dos.  

 In  1991, I had remarried, moved again, and had a new teaching job. With a 10W solar panel, the laptop could be used,  and I tried various editors to develop a simple database---a list of animal names with descriptive text.   I needed a programmer's editor that did not mangle the text with control codes.  A linguist from the University of Hawaii sent a demonstration copy of MultiEdit, which did some of what I wanted. But to learn how to type diacritics, I would need to purchase the manual, for over 300.00. On my teacher's salary, this was impossible.  

 At the time, I had been reading Infoworld, and , ran across  a short article buried a few pages in, about the Free Software Foundation. I knew nothing about this, or about Free Software, but the name said it all. I wrote a letter to the FSF explaining my project, and asking for help for an editor. A few weeks later, I received a package with about 13 3-1/2" diskettes, a package that would change my life.  (Whom should I thank?)  On one disk was "Demacs," an implementation of Emacs for the Windows ecosystem, in addition to a number of Unix text tools ported to Windows, by Cygwin, and Unix Text Tools---sort, ptx, grep, find, and other---ported for Windows 3.0.   

 Demacs was a well-functioning port of Emacs to Windows. One of the first things I noticed was the existence of a Tutorial, a genius introduction.  And this was the "self documenting text editor."  Within minutes I was sufficiently able to navigate the interface and to work with text.  I developed a crude understanding of Emacs Lisp, and wrote several low level helper functions for my needs. This software was beyond any dream. 

  Shockingly, in this age of continual modernization, most of the tweaks I developed to my first emacs files still function on today's Emacs 30+, in 2026, not the case for files written on early windows Word versions.  Happily, over the next couple of years, the Gnus Bull, the bulletin of the Free Software Foundation arrived every so often in my post office box. In one of these bulletins, fascinating reading for me at the time, was an article about Unix work-alike operating systems, Linux and FreeBSD.  Later on, on an extended medical visit to Guam, I was able, through generous assistance of the owners of the ISP Kuentos Communications, I was enabled to download Slackware and install it on my Toshiba Laptop.

This meant I could install Emacs directly. The Unix text tools were available directly as well.  My needs are simple, compared to many---if not most---Emacs users, many of whom are programmers comfortable with generating elaborate and extensive code. As my project grew, Emacs was the essential ingredient. Later on I learned of the existence of LaTeX. Eventually, my lexicon was published from camera ready copy generated by LaTeX in the journal Micronesica.  With the help of a programmer at the University of Hawaii Linguistics Department, an Emacs guru, I was able to write an emacs lisp program to "alphabetize" Chuukese in an arbitrary order.  Many other scholars and programmers have helped along the way, on mailing lists, or through lengthy correspondences.   Early on, Professor Hsu at UH mentioned the Band Format---a free form database used by linguists for lexical data.  I could not afford the tools he offered, but I was able to mimic some of the needed facility using Emacs lisp.  

 I was able to implement a tool to convert each record of the band format database into LaTeX code for a single entry, that was typeset as camera ready copy for the ultimate publication.   

 For this carnival, the take home messages for Newbies and scholars was that the Tutorial can provide, in an hour or two, a tremendous introduction to the keyboard use for Emacs.  And secondly, I think it's important to recognize the number of Emacs experts who are willing to assist in such an endeavor as mine.  I have not resorted to a starter kit.   

Wednesday, April 8, 2026

Setting file times in Emacs from Time-stamps in file, or "Created " at start of a file: Generated through Google AI Summary

AI is problematic, but the sheer convenience of it has grabbed my attention.  For tweaking my GNU/Linux system, AI summaries of Google searches come in a few seconds, and they sometimes save hours of research.  It is helpful that low grade links to sources are often provided.  I balk, however, at the damaging infrastructure involved.   

 A recent adventure has been developing a useful emacs lisp tool to change the metadata time stamps of a file from emacs' Time-stamps, when they exist.   

Today I realized I had copied files from another system without specifying to save file metadata.  Could I set the file times of files with timestamps using the "Time-stamp: < >" utility in Emacs?   I searched, and, I must freely admit, I asked for more info from the AI helper of Google Search.  After about 7 or more rounds of providing feedback about  a regular expression that did not work, these functions were provided.  Wow.   The last two of these is probably sufficient for my needs.  I will change the names.

 

(defun my/set-file-time-final ()
  "Sets file time by finding 'Time-stamp' and extracting only the date/time.  From Google AI."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    ;; 1. Find the literal word "Time-stamp" anywhere (ignores #, brackets, etc.)
    (if (re-search-forward "Time-stamp" nil t)
        ;; 2. Look for the first date pattern (YYYY-MM-DD HH:MM:SS) that follows it
        (if (re-search-forward "\\([0-9]\\{4\\}-[0-9]\\{2\\}-[0-9]\\{2\\}[ T][0-9]\\{2\\}:[0-9]\\{2\\}:[0-9]\\{2\\}\\)" nil t)
            (let* ((ts-string (match-string 1))
                   (parsed (parse-time-string ts-string))
                   (lisp-time (when parsed (encode-time parsed))))
              (if (and (buffer-file-name) lisp-time)
                  (progn
                    (set-file-times (buffer-file-name) lisp-time)
                    (message "Success! Mtime set to %s (Username ignored)" ts-string))
                (error "Parsed '%s' but system couldn't use it" ts-string)))
          (error "Found 'Time-stamp' but no YYYY-MM-DD HH:MM:SS format found after it"))
      (error "The word 'Time-stamp' is missing from this file"))))

(defun my/dired-set-times-from-stamps ()
  "In Dired, run 'my/set-file-time-final' on all marked files. From Google AI."
  (interactive)
  (let ((files (dired-get-marked-files)))
    (dolist (file files)
      (with-current-buffer (find-file-noselect file)
        (condition-case err
            (progn
              (my/set-file-time-final)
              (save-buffer))
          (error (message "Skipping %s: %s" file (error-message-string err))))))
    (revert-buffer) ; Refresh the Dired buffer to show the new times
    (message "Done processing marked files.")))

(defun my/dired-set-times-from-stamps ()
  "In Dired, sync system times for marked files. Files without stamps are ignored."
  (interactive)
  (let ((files (dired-get-marked-files))
        (skipped 0)
        (updated 0))
    (dolist (file files)
      (with-current-buffer (find-file-noselect file)
        ;; Use condition-case to catch files that don't have the stamp
        (condition-case nil
            (progn
              (my/set-file-time-final)
              (save-buffer)
              (setq updated (1+ updated)))
          (error (setq skipped (1+ skipped))))))
    (revert-buffer)
    (message "Done! Updated: %d | Skipped (no stamp): %d" updated skipped)))

(defun my/set-file-time-final ()
  "From Google AI summary session after Google search.  Sets file time from header. Handles: Created YYYY-MM-DD Day HH:MM:"
  (interactive)
  (save-excursion
    (goto-char (point-min))
    ;; 1. Search for 'Time-stamp' or 'Created'
    (if (re-search-forward "\\(Time-stamp\\|Created\\):?" nil t)
        ;; 2. Match the date (YYYY-MM-DD) 
        ;;    Then skip any day-of-week (like Tue)
        ;;    Then grab the time (HH:MM or HH:MM:SS)
        (if (re-search-forward "\\([0-9]\\{4\\}-[0-9]\\{2\\}-[0-9]\\{2\\}\\)[^0-9]*\\([0-9]\\{2\\}:[0-9]\\{2\\}\\(?::[0-9]\\{2\\}\\)?\\)" nil t)
            (let* ((date-part (match-string 1))
                   (time-part (match-string 2))
                   (ts-string (concat date-part " " time-part))
                   (parsed (parse-time-string ts-string))
                   (lisp-time (when parsed (encode-time parsed))))
              (if (and (buffer-file-name) lisp-time)
                  (progn
                    (set-file-times (buffer-file-name) lisp-time)
                    (message "Success! Mtime set to %s" ts-string))
                (error "Parsed '%s' but couldn't use it" ts-string)))
          (error "Found keyword but date/time format was unexpected"))
      (error "No 'Time-stamp' or 'Created' header found"))))

(defun my/dired-set-times-from-stamps ()
  "From Google AI summary session.  This calls the function my/set-file-time-final.  In Dired, sync system modification times for all marked files
based on the 'Created' or 'Time-stamp' lines inside them."
  (interactive)
  (let ((files (dired-get-marked-files))
        (updated 0)
        (skipped 0))
    (dolist (file files)
      ;; Open the file in a buffer without switching to it
      (with-current-buffer (find-file-noselect file)
        (condition-case nil
            (progn
              ;; Call your working 'within-the-file' function
              (my/set-file-time-final)
              ;; Save the buffer to ensure metadata is flushed to disk
              (save-buffer)
              (setq updated (1+ updated)))
          (error 
           (setq skipped (1+ skipped))))))
    ;; Refresh the Dired buffer so you see the new dates immediately
    (revert-buffer)
    (message "Batch complete: %d updated, %d skipped (no stamp found)." updated skipped)))

(with-eval-after-load 'dired
  (define-key dired-mode-map (kbd "T") 'my/dired-set-times-from-stamps))

Wednesday, March 18, 2026

Shinestacker for focus stacking: Hurray!!

 This is the first image I processed using Shinestacker.  It's a remarkable and technically advanced piece of open source software.    This software is beyond my skill level, but works incredibly well with defaults.  

 

Sufficeth to say it was very simple, after a somewhat convoluted---but not difficult---installation on CachyOS, to process this stack of 40 images.  I have a CPU with 27 or 28 processors; and this software even took advantage of my aging Nvidia GTX-1050 video card with a mere 3GB of video ram.  25X objective, mounted in UV-curing glass glue from a hardware store.  Too bad about the detritus.  Processing took a few seconds.  


 

Ossicles of sea cucumber Holothuria atra, mounted in UV-curing glass glue.

 UPDATE (2026 May 18)

  Updated Shinestacker this week,  Incredibly, it can accept Canon EOS *.cr3 raw files.  Neither Fiji (unless by trickery), or Helicon Focus can open these files.  The developer of Shinestacker stated that tiff files work better on SS, which apparentely was a major impetus for accepting cr3 files.  It works well, but required more resources.  My 26 core intel i7 cpu, with 32G of ram, and 128G of swap space, works well; shinestacker utilizes all cores, I believe.
 
Some images have shown blotchiness.  In one case, even using .cr3 files did not solve this.  These are microscope images, which may be a factor.  The focus stack of images of ossicles, above, used very few shots, a rather coarse Z sampling.  

The ease of using this software is a huge upside.   Some experimentation will be required to figure out advanced settings, ie, what kind of images do well with pyramid stacking, and which work best with depth masks.  I had some problems on some images; it seemed, perhaps, that turning off balancing may have helped.  These images are fairly consistent in illumination.
 
Hurray for  this software!!! 
 
 

Actually, Wayland is usable. I am leaning on Sway on my laptop.

 Just to bring the conversation up to date.  It is easier to use sway on this laptop---a Lenovo X1 Carbon gen. 9---because it has a HiDPI display.  I3/X11 are problematic, requiring numerous configuration tweaks.  Hyprland is easier, but less compatible.  

 

My personal customizations for i3 are not completely compatible with Sway.  For one thing, I have adopted dropdown windows for a number of applications on i3, which is sometimes problematic, but usually works.  Kitty works ok in this way on Sway.  Emacs, not so much.  The following applications have bindings in my i3 configuration, and work well: orage (calendar); xephem; R).  Emacs is ok, but tricky.  Kitty too can be tricky.  This manner of using i3wm is fantastic for my workflow.  I also have had kmag on a dropdown keybinding, which is useful.   

 I have, as of May, 2026, not made much effort to get all of these utilities working on Sway.  Dropbox's hot icon does not work on i3, so far, for me; on Sway it does work with swaybar, an excellent bar.  

 Tiling WMs work for me.  I3 is easy to configure.  Sway can run ok using the i3 config files, except for edge cases.

 

 

 

 

 

 

 

 

 

Thursday, February 5, 2026

Trying out Sway, an i3wm workalike for Wayland

 A good move I recently made was to sell my MacBook Pro, M2; and purchase a Lenovo Thinkpad X1 Carbon, several years old.  One of the best features of this new laptop is a HiDPI (High Dots Per Inch) screen with a resolution of something like  3200x1800.  It's a remarkable display for other reasons, which I will not go into here.  The blacks are incredible.  

 With my usual Manjaro i3wm, this means that the type is almost unreadable.  With the guidance of several source on the Internet, I was able to cajol the system into somewhat useful settings that most software works fine with.  Along they way, several references passed me by about Wayland, a newer graphical setup that X11.  

 

I am reluctant to make huge changes to a long time stalwart system, but finally I have begun to experiment with Sway, a workalike system to i3, and Hyprland, a fancier tiling window manager, working on top of Wayland.  Both, I am happy to report, work nicely with this  HiDPI screen.  

 

So I installed Manjaro's Sway edition.  So far, so good.  Now I'm at the point of trying to configure pop-up/drop-down floating windows for Kitty, Emacs, R, orage (probably impossible, but this is the best little calendar app I have found.  Once I have set the month, and popped it back up, the same month is show.  Very useful for calendar work), and a ham radio clock, that is actually of little use.  

HiDPI is not a problem.  

 

When I install a new distro, one of the first things I do  is set the user id to "1004", as well as the groupid.  Consequently, the home directory of my previous setup and user are completely accessible, because this was the case for the previous login as well.  I never duplicate the login, out of worry that the dot file tweaks will be incompatible with anything different.  

 

So far, so good.

 

Thursday, September 4, 2025

Tools of Excellence: Cb2bib, Yazi, Sioyek, Entangle, XnViewMP, Emacs,...: WOW!

In the summer of either 1982 or 1983, I enrolled in a summer school class at UCSB offered by the College of Engineering on Computer Architecture.  At about this time two of my regular classes required the use of Wordstar, on a mainframe terminal.  I had been required to write a program in Fortran for a math course; now, in this Computer Architecture class, I wrote some minimal code in PDP-1170 Assembly Language.   Intel's early microprocessors---the engines that  would spark a revolution in Personal Computing---had come into being.   I had enrolled on a Pass/Fail basis, fearing that it would be a technical challenge.  I had enrolled later in an amazing  Invertebrate  Paleontology course, also Pass/Fail.  Both times, I would have earned an honest "A."         Yet another Lesson learned.  

 The theory behind digital computers, as it turns out, is straightforward and fascinating, logical and understandable.  I later came to realize that these systems, invented  by humans, reflected human logic; they are non-complex, a fact I understood more fully after being introduced to Molecular Biology and Genetics, and the extreme complexity of the Central Dogma  By the end of the short summer session, I had a tentative grasp of the nature of digital computers.  The graduate student instructor took me aside and advised that the knowledge I had gained in this brief introduction would be valuable in the future, and I would be able to advise others about what computers can do.  

 This advice could not have been more applicable when I moved to Chuuk Lagoon, and started a project collecting animal names in local dialects of the islands.  At the University of Guam Marine Lab, I was exposed to early Personal Computers, the IBM XT and IBM AT.  My basic knowledge of the workings of computing systems provided useful perspective for solving problems.  I eschewed the commercial/proprietary solutions, the high priced computer programs that would make the entrepreneurs wealthy.  

 I learned what I could and scrounged software of many kinds.  Except the big brands.  There are always other solutions that work better; those behemoths aren't worth the cost to society.  I believed this then, and I believe it even more today.  

 My Toolbox

Previously, I have written a little about Yazi and Sioyek.   They, like Cb2Bib and Entangle, are functionally remarkable tools.   Among Free Software users, they are not the most popular.  I also rely upon the highly functional, low overhead foundation, the i3 tiling window manager.  Emacs is a mainstay; like these other tools, the simplicity and functional design of Emacs makes it run circles around most proprietary software.  Windows is dumbed down; the stress of using such a dysfunctional tool might be likened to trying to build a cabinet using a jackhammer; a poorly made jackhammer at that.   I use LibreOffice from time to time, and some other more mainstream tools.  They are fine; but the workhorses in my toolbox are more special. The packages I am writing about today are truly special, and enabling. 
 
I as initially attracted to Free Software for financial reasons.  The story is too long to recount here.  But as I have come to understand the GNU Project, and the grass-roots nature of the development of these tools, it became clear to me that the Free in Free Software actually refers to freedom, not free beer.   
 

Sioyek

The PDF reader Sioyek caught my attention relatively recently, with its plain exterior, focus on research, and its  hidden gems.  For a couple of days, I returned to sioyek, after setting it aside a while; a closer reading of the documentation has given me a new appreciation of it's wonders.  And today, after some digging, I was able to configure yazi to open files in Sioyek, or Evince, at will.  Gradually, the levers and buttons of sioyek are revealed.
 
The documentation is relatively straightforward.   But I struggled with configuration.  Later for that.  PrintHere are some of the more interesting features:
  • A keystroke sends the text at the cursor to Google Scholar (GS), or another tool I am unfamilar with.  A browser opens, Google Chrome.  Keystrokes are available, either standard or by a plugin, to download the pdf from GS.  
  • Sioyek remembers bookmarks of two or three kinds.  A keystroke causes display of a list of bookmarks in either the current file, or all pdfs sioyek knows about.  
  • One of these "bookmark" functionalites is called "highlighting."  Selected text may be highlighted.  The keystroke for highlighting must be followed by a single key, which represents the type of material.  So someone like me, whose mind travels in so many directions, can keep track of various threads of interest.  If several files are highlighted in this way, retrieval of highlights can optionally display all highlights of a given type in all files (globally).  
  • The Global retrieval feature holds for bookmarks as well.  So if I am reading four PDFs that relate to each other, I can bounce around.  And selections can be copied to the system clipboard.  You know where this is going.
  • Portals are something I don't yet understand.  I position may be set as a target for a portal, so as I'm reading I can refer back to a certain specific piece of text, diagram, table, whatever.  Backspace brings me home.
  •  I was able to print.
  • Sioyek is mainly keyboard driven; but scrolling with a mouse is possible.
  • A "visual mark" (or whatever it's called) underlines current text if that is wanted; and this mark can be manipulated in various ways.  More to learn hear.
  • Zooming is great.
  • In PDFs text can be selected and various functions are possible.  A click on a figure number will take me directly to that figure.  This built-in feature seems almost intelligent, and it reminds me of snarfing references in Cb2Bib, a remarkable feature.  
  • Synctex mode allows the latex source to be opened and followed when reading a pdf.  Wow.

Some thought went into this tool.  

 

Yazi 

Yazi is a console based file manager with superpowers.  I have gotten into the configuration details far enough to make it possible to choose which pdf viewer is desired.  Sometimes, I think sioyek may be too far out, for some things; so I set up to be able to open in either  evince or sioyek.  Shift-O opens a minimal menu; included here is a button to open it in a folder.  I think some more tweaking is needed to identify low-overhead tools like qutebrowser for Google Scholar searches, and a minimal Graphical File Manager with lower overhead.  My laptop lags seriously, but it's a very old laptop.
 
Yazi works swell with kitty.  Previews of jpegs and pngs, and possibly tiffs (with some tweaking) is extremely fast, and a plugin "zoom" feature enables the image to zoom to the full height of the window.  Image and PDF previews are useful.  
 
Bunny is an amazing bookmarking tool.  
 
A tweak in .bashrc (or .zshrc) makes it possible to open yazi by typing 'y'.  As a bonus feature, when one changes directory, then types "q" to quit, yazi leaves the console in the last folder visited.  
 
This tool is undergoing rapid development, and plugins have been known to fail after updating.  
 
I use this for editing filenames, with tags.  A filter function ("f") then makes possible isolating files with a specific tag, marking them all, creating a directory, and dumping all these into a directory.   
 
Cb2Bib
 
This is an extremely useful tool, to me.  I think I'm the only one who uses it in the way I do, to write annotations to records in my bibtex database, and print out annotated bibliographies on demand.  I write, for example, a call number of a book, and notes on pages, compile it into LaTeX, and print.  Very convenient.  The developer was kind enough to explain how to modify the code to incorporate an "annotate" field in the bibtex entry, and also modify the tools that generate a pdf, so an annotation is formated together with the citation.  
 
In this space, I hope to present instructions for the benefit of others.  

Cb2Bib also automatically picks up citations that are highlighted in other programs.  Also, one can copy a bibtex citation from Google Scholar, that is instantly picked up and incorporated into the database associated with the program.  

This program does much more than I am able to understand, but what I can understand, and do---with the help of Pere Constans---is amazing.  
 
This is another reason I don't stick to the commercial / proprietary software packages.
 
Entangle
 
More to follow. 
 
 


Saturday, July 12, 2025

sioyek and yazi

I have set Sioyek as the default pdf viewer for Yazi; actually, probably for the entire system.  

 It was *relatively* simple to do this, after installing xdg-utils-mimeo.  I will not go back through that here, but it's not hard.  

 Sioyek has no native capacity to print.  This is a potential show stopper.  I have attempted to set this up from instructions from some random internet post.  The location of the user config files is in doubt, however.  By "reports," config files should be found in ~/.config/sioyek; however, other reports indicate .local/share/sioyek.  I did install a rather decrepid tool, a gtk-print tool (name?).  My config does not work yet.


Another potential downside relates to the use case for Sioyek, as a pdf annotation tool for research.  When a file is opened for the first time, sioyek indexes the file; this makes searches incredibly quick and useful.  Will this result in cluttering the system after a large number of pdfs are opened?  Probably another lightweight pdf viewer should be established as the default, to streamline the process of reviewing random files on the fly.  

Sioyek is uncluttered, as far as GUI.  There hardly is any GUI.  A fairly extensive set of utilities is tied to keystrokes.  Some of them are not exactly intuitive, at least to me.  


Some facilities:

  • Bookmarking
  • Search in google scholar (may not work right)
  • gg -> go to page 1
  • G -> go to last page
  • rt click -> mark line (not sure this is working with a track pad)
  • highlighting in various colors
  • search highlights
  • "Smart Jumps" to figures or references, with or without links: this didn't work out of the box, for me.
  • mark a location, give character as a name for the mark.

 

Yazi is turning out to be extremely useful.  I use it multiple times a day.  Some useful tweaks:

  • set a much larger preview size, then
  • install a preview to Toggle full view of the preview
  • somehow a special tweak was needed to view tiffs (this wasn't the case when I first tried yazi)
  • bookmarks.  yabm (Yet Another Book Mark) is great.  Another one was incompatible, AFAICT.
  • Zoom plugin
  • In a tips section of the web site, is recommended to install some code in the rc file of the shell, which allows to start with "y" and that leaves the terminal in the last used directory, when quit with "q".  Extremely useful.

A problem: not really a big one: yazi is undergoing rapid and intense development, so incomptibilities arise frequently.   Especially with plugins.

 

 


Some recent useful software additions

 A brief list for now:

  • dysk : concise "df" and graphic
  • sioyek: interesting  document reader.  Keyboard driven, mostly, maybe. Designed for academic papers and textbooks.  
  •  bpytop: A feature-ful process/memory/cpu monitor, text based; similar to top or htop.
  •  

Monday, May 19, 2025

Notes on Yazi

 Yazi is good.  Real good.  I used it over ssh to cull a directory on my second system; it worked as I had hoped, except, perhaps, for color theming.

 Knock on Wood.

 Viewing/reviewing

One of the needs that yazi has fulfilled for me is rapid viewing.  I can only imagine that should I ever be able to upgrade my current machines (RAM and CPU, as well as Storage), viewing would be even more friendly, especially for images.  The ability to flip through a directory of images, mark, and move, copy, or delete marked files, has made a huge difference to my workflow.  The only other tool that is useful in this regard is xnviewmp.  I use a fairly simple set of commands; probably if I mastered tabs and other complexities and plugins, yazi would be even more helpful.

 

By the way, the terminal "kitty" has wonderful facilities built in to work well for viewing images.   I prefer the black background and contrasty color scheme to any other.

 Ranger was good enough, until it wasn't.

Copy, Paste and move and Notes on plugins.

 [Cautionary note: I have lost files through carelessness here, and yazi has some gotchas to be navigated around.]

This is pretty much self-explanatory.  I will try to address this topic in detail at some future time.  For now, I'll mention a plugin that has helped speed these processes: 

  • yamb is a bookmarking plugin that works well.  In the absence of a much needed multi-step backup command, this does a good job when sorting through files and moving or copying them to another directory.  I have gotten around the mentioned limitation by defining  bookmarks for the current from directory, and the currently interesting goto directory,  I define a set of book marks to frequently used directories, as well.  This is an easy tool to master.  Like other plugins, issues have been experienced after updating yazi (it is under fairly intense development).  
  • toggle-pane: This helps when viewing both pdfs and images.  It is possible to configure the mazimum size of displayed images (I'm not certain whether this works for pdfs) by putting this code into ~?.config/yazi/yazi.toml:

                       [preview]

                      # Change them to your desired values
                      max_width  = 1800
                      max_height = 1800
  

       Plugins are found on a resources link on yazi's web pages.  Other sites exist with plubins.

 

Filtering

Filtering is a wonderful way to sort.  It has been even more useful to me, because of a filenaming scheme I have adopted.  I stole this idea from Protesilaos's Denote system.  I have not found denote very useful, but the concept of tags at the end of the filename, before the extension has become a useful tool allowing me to sort files more rapidly.  Here is an example.

    Jones--2060--JargonInBiology__taxonomy_nomenclature_zoology.pdf

I have started avoiding underscores in filenames ("_") in favor of using them in tags.  Interestingly, I saw a web page or thread somewhere about tagging styles, and this is one of several.  I used to use a 10 (or so) character at the beginning of lines in a bibliography, where each position had a meaning.  So anything related to crabs would have "c" in the third position (12ctx8urt-Anon-Crabs of the world) would be sorteable by sorting by the third position.  The Unix (GNU/Linux) sort command worked excellently.  

 With filtering  in yazi, one can narrow down a large folder/directory with pdfs about taxonomy: "_taxonom".  Prot has demonstrated the use of regular expressions to sort even more specifically in, I think emacs dired.  To avoid filtering for "_" in filenames, one would theoretically use a regular expression requiring a "__" (double underscore) somewhere before the "_:" single underscore.  This is probably easy.  Filtering is easier, I think.  It may be that emacs would be better, but so far, aside from a couple of learning experiences, Yazi is working well.

Moving around 

In either the getting started or tips section of the yazi main web pages, is a suggestion that makes yazi more useful.  

Inserting this code into .bashrc (and I think .zshrc as well) makes things easy.

 function y() {
    local tmp="$(mktemp -t "yazi-cwd.XXXXXX")" cwd
    yazi "$@" --cwd-file="$tmp"
    if cwd="$(command cat -- "$tmp")" && [ -n "$cwd" ] && [ "$cwd" != "$PWD" ]; then
        builtin cd -- "$cwd"
    fi
    rm -f -- "$tmp"
}

Then, when in a terminal, type "y" to start yazi.  After moving to another directly, quitting with "q" leaves the terminal in this new directory.

 

A Caution or Two 

 Plugins are often not mutually compatible.  I have tried several others, but some collided with these few i do use.  

Some plugins iexplicably use keybindings that are assigned to other functions by default, and it is not always apparently what key bindings may be used.  <F1> may display a help screen, which may be filtered.  Filtering rocks.  Bookmarks rock.

 

2025 May 19

 

 

             

Monday, February 10, 2025

Unix's and GNU/Linux's Unknown Superpowers: ptx, sort, and grep

The Unix operating system is wonderful.  GNU/Linux is wonderful.  Let me make this distinction, before discussing the aforementioned three tools: GNU is Not Unix. This is a story of how capitalism limits the human spirit, in order to extract financial gain.  

 Before writing about the remarkable tools provided by Unix---and now, the GNU computing system (one of the scions of which is GNU/Linux)---one may be forgiven for visiting certain concerns, including the generation of incomprehensible wealth by a few men who now seek to control our lives in ever more of their detail.  I suggest that this wealth was made primarily through marketing, on the backs of the ingenuity of other men, some of whom were out-maneuvered and left behind by certain well known business men.  

It came about, during the early days of computing, long ago: a gifted programmer by the name of Richard Stallman worked at MIT Artificial Intelligence Laboratory (MITAIL), along with other highly capable computer programmers.   Events during that period led to unforeseeable changes, as a behemoth technical-industrial complex grew beyond all imagination, based on deft slight-of-hand and shrewd business moves, as entrepreneurs snatched up computer code, patented and copyrighted it, and concealed it from hapless consumers.  Shrewd manipulations of programmers and code enabled a few men, in less than 50 years, to accumulate over half of the wealth of the financial world.  

This they were able to do by convincing consumers not to share, with the complicity of the legal "intellectual property" infrastructure; to violate the golden rule that many had cherished since childhood.  Make many copies of a useful computer program for a few pennies each, and set prices  arbitrarily high.  The consumer will be so mesmerized by the utility and flashiness of these programs that she will believe you are doing her a favor, selling it so cheap.  Convince governments to sign lucrative contracts.  Update products at speed, and charge for the updates.  Sue public schools when they did not update to newer licenses.  the profits would be almost unlimited.  And, now, change the business model and lease software, so leverage even greater control, and much greater profits. Never do unto others anything without transactional benefits to self.  Above all, do not share a computer program, even though they are useful.  This was a massive wealth grab. 

Tim Wu has written about the  entrepreneurs who built incomprehensible wealth by these same strategies: seize control of the media channels, prevent newcomers by various and nefarious means, and charge arbitrarily high rates for the use of this now valuable commodity:  information.    Radio. Telegraph, Telephone, and Television have been used to manipulate the public, by employing choke points---selective filters---to manipulate elections, gain power, and sell snake oil.  The political landscape has been altered in the interest of gaining more power, prestige, wealth, and control.  It's OK, these persons believe, to lie and deceive.  Today's politicians have gained success by following a model of radical "gas lighting," and in doing so have built empires  of corruption and greed.  

You may know this story. 

 Richard Stallman has been painted under the dark lights of deception as an extremist, even among other adherents to Free Software.  This is how the insidious tactics of industry and politics can influence our opinions to their own benefits.  The Petroleum Industry, in a similar manner, convinces us that global climate change is a hoax, that CO2 is fine.  Our politicians convince us, who are eager to consume their distorted and deceptive messaging, of these things, and many others.  We know they are lying, but the steam engines of their corruption are traveling down the tracks with such unrelenting force and momentum that they seemingly have taken over our own minds and hearts.  

 This is about one little part of the story. 

The MITAIL was at the forefront, in those beginning days of computing.  Richard Stallman was there; he noticed that the software interface of their very expensive printer was inefficient, klunky.  In the hopes of improving this interface, he kindly offered to the company that owned the patents,  to tweak the software and make it work better.  The manufacturer, however, was unwilling to open up the code behind that software, fearing loss of control and profit if it ware revealed.  

This is an important story, but this is not our story today.  Suffice it to say that Richard Stallman decided to do an end-around, in the finest traditions of human cooperation and sharing.  The Unix Operating System, owned by Bell Laboratories (AT&T), was an outstanding work of programming, a thoughtfully produced system of software, an interface of great utility, with numerous subprograms attached to a central kernel, making a whole that by far exceeded the sum of its parts.  But AT& T owned Unix. In the capitalistic world, this was a fantastic money maker.  But the cost of ownership was onerous, making it impossible for the average computer user to purchase.  And as Stallman had already learned, the impediments that proprietary software posed to even the best programmers presented various difficulties---even beyond simple ownership.  The experience with a printer had taught that lesson.  


And what about sharing these wonderful tools?  Stallman recalled an earlier time when the many talented programmers and users at the MITAIL had freely shared programs they had developed.  As a consequence of this open access to the universe of such software programs, even greater programs were spawned from the synergism...  But now many of those programmers had been enticed to leave this world by unimaginably lucrative contracts.  But  these contracts had been concealed within a trojan horse: the non-disclosure agreement.  The code, they promised, they would not share.  And even the end users, who purchased such programs today as Photoshop, Lightroom, games, typing tutors, graphics printing programs....  ad infinitum, are coerced into not sharing the software they purchase.  The industry is serious about this limitation.

So Richard Stallman reached out on the internet to propose a collaboration of programmers to build a new computer system, a free system, that could be "shared without dishonor."  Remember how Unix comprises a set of software units that are hook in to a kernel?  Those units include various commands: copy and move files, for example.  Even typing on a keyboard is handled by tools hooked into the kernel.  The construct, from a plethora of these units, is an Operating System (OS).  Such operating systems as the proprietary MS-DOS, Windows, and MacOS are example of proprietary OSes.  It is forbidden by the proprietors of these OSes to share the programs.  At great expense, these companies have constructed elaborate schemes to make it difficult or (they hope) impossible to share these programs.  

 Do you see where we are going with this?  This massive construct, including now Microsoft, Facebook, Apple, Google, Oracle, Adobe, and others, has used a strategy of concealment, of forcing others to not share, to build huge empires, and horde wealth.  Intimidating the consumer, for example by suing Portland, Oregon's public schools to force purchase of contracts for every copy of windows that had not been updated, as "required" by the manufacturer.  

But GNU is Not Unix.  Richard Stallman recognized Unix as a wonderful model for an OS.  If thousands of programmers all over the world, connected by the Internet,  were to each write one or a small number of   utility programs, mirroring those that perform the functions  of  the Unix programs, while avoiding even any appearance of copying the Unix code, a new operating system could be built up through community action, a citizen-built computing system.  This system was successfully built.  Even a protracted legal challenge by the owners of Unix, SCO, many years later, could not find any copied code in the GNU/Linux operating system.

When I was isolated in Chuuk Lagoon, I had been conducting research that required access to computers.  My family donated a laptop computer, but the Windows OS was clumsy, and, to cut to the chase, could not help me with my project.  An editor that had been sent to me by a linguist at the University of Hawaii was capable, but it was "cripple ware," in a sense, and without the manual---which sold separately for hundreds of dollars--I could not proceed.

 

 The Free Software Foundation (FSF) responded to my request for an editor, sending a version of the Emacs editor ported to Windows, along with 14 disks of Unix tools, also that would run on Windows.  This was just what I needed.  Among these tools were a number of Unix text tools, opening a window into a new world of possibilities.  Very important to me were sort, grep, and ptx.  These, as far as I am concerned, are core Unix (hence Linux) superpowers. 

sort

Windows had a sort utility, but it was extremely limited.  In fact, the more I learned of the tools granted to me by the FSF, the better I understood the fraud that had been perpetrated on the public, corporate world, and governments by Microsoft.  I can liken the difference between the facilities provided by Unix (and later by Linux and FreeBSD) is comparable to the difference between a set of cheap carpenter's tools, and professional quality tools.  GNU sort is the Cadillac of sort programs, the Rolls.  

Later, on Guam, I was able to download a copy of the new Linux OS on Guam.  It was then that I discovered liberation.  From that day, in the early 1990s, until today, in 2025, I never never felt comfortable on a Windows computer.  At that time, one of the biggest differences was that Linux was capable of doing true multitasking, while Windows only could do one operation at a time.  With GNU/Linux I could type a letter while printing a different document, and running another process in the background.  Another difference was the Linux did not crash.  Let me repeat, it never crashed or locked up.  Windows regularly did so.  

grep
 
 I will not explain grep.  Today, in the 2020s, one can run grep on Windows machines.  Suffice to say that grep is a tool that enables searching through a set of files for a search term, and printing to the screen a list of matches. With pipes (signified by the character "|", one can direct the output of grep to another program, like sort.  Then, from sort, one may direct its output to less, a better file browsing tool than Unix's more, a tool that pages through a file.  To do this one would type:
 
    $ grep <filename.txt> | sort | less

 

ptx

 My need was specialized, as I was dealing with lexical text, lists of animal names.  To weed through these, I found ptx was incredible.  Ptx means "permutated index."  What follows is a small segment of output from a search of a text file about flowers and seasons of fishing. 


  /Luma, altitude 100-125 meters,   flower, August 4, 1921, Garber no./
      /\section{Burrows: a different   flower, and palolo.\hfill{}\textsc/
     specifically, he does mention a   flower.
     with certain things. There is a    flower called bust ginger;/    /time
      \textit{Erythrina\/} sp. as an      indicator species.            /about
        /constellation as a seasonal     indicator, he refers to this/
   a resident of Wonipw,/       This    indicator would only be useful to
   the/    /solar alignment as a key   indicator of the   emergence of
      /\hfill{}\textsc{whale:season:    indicator}} \label{sec:org5e260c7}/
     /tree fruits are ripe, it is an        indicator that whales are running/
Samoa/    /was used as a seasonal  indicator on some islands (e.g.,
  /2015, Trees of Yap:  sometimes   indicator of seasons} \label{sec:/
   . \textbf{Sometimes used as an   indicator of season.}       /flowers
   a resident of Wonipw,/       This   indicator would only be useful to
    /variegata\/} flowers in the dry   season, and has been mentioned (/
      and that it flowers during dry   season.       /of year of flowering,
      /these two periods the iéboua   season; the \%\%    season of the/
       /the iéboua season; the \%\%   season of the sun or heat iéboua-/
    /or heat iéboua-délat, and the   season of cold iéboua-tsiam.


tldr

 

Tldr (too long didn't read) is a modern innovation that summarizes the man page for a command.  In Unix, it was specified that who ever wrote a program must provide a "man": page, a manual to its use and options.  Here is output from tldr:
 
$ tldr ptx

  Generate a permuted index of words from text files.
  More information: <https://www.gnu.org/software/coreutils/manual/html_node/ptx-invocation.html>.

  Generate a permuted index where the first field of each line is an index reference:

      ptx --references path/to/file

  Generate a permuted index with automatically generated index references:

      ptx --auto-reference path/to/file

  Generate a permuted index with a fixed width:

      ptx --width=width_in_columns path/to/file

  Generate a permuted index with a list of filtered words:

      ptx --only-file=path/to/filter path/to/file

  Generate a permuted index with SYSV-style behaviors:

      ptx --traditional path/to/file 

Saturday, October 12, 2024

The Free Software Foundation (FSF) is a beacon

I just stumbled upon a statement on the website of the FSF (fsf.org) about the appointment of three new board members of the organisation. I am impressed. The FSF has been on my radar for many years, as I have followed developments in the Free Software world, and used Free Software for my own work. I have been a devotée of free software for over 40 years, as a user. These people at the FSF, as well as the Electronic Frontier Foundation, and GNU: what's not to like? Who is more committed to the concept of free software, free information. freedom from repressive copyrights and patents? Fantastic! The FSF!

How I got into Emacs (and Free Software)

This is a long tale, not confined to Emacs alone. This is, indeed, a story about why I am a confirmed Free Software advocate. This has been...