Searching by sender in notmuch

Hello everyone,

I’m not sure if I had mentioned this before, but for quite some time now, I’ve been managing email with Emacs and notmuch. I’ve always thought tags were much better than folders because they’re more flexible for classification. It took me a lot of effort to adjust the tags I use and how I handle them. This scenario used to make me quite anxious:

I receive an email that I tag and classify.
I get another email from the same thread, and I have to classify it again.

Why? Since it's the same thread, every message should have the same tags. It’s true that the thread displays all tags, but sometimes I’d lose messages in searches, or some would be archived while others weren’t.

Another thing that bothered me was not having a quick way to search messages from a sender in the search screen. It's super useful for cleaning up emails by sender.

But that’s why I use Emacs — to quickly fix these kinds of things ☺️

Filter by sender

Let’s start with the easy part: the search function by sender. Here, we need to address a couple of issues. In notmuch-search-mode I haven’t been able to find a function that returns the sender of the current thread. There’s a function notmuch-search-find-authors which returns the name shown in the view, but not the address. Maybe I could use that to search in the address book or something, but I found another method first: running the notmuch address command with the thread-id and some extra parameters.

notmuch address --format=sexp --format-version=5 --sort=oldest-first  thread:XXXXXXXXXXXXXXXXX

Parameter	Description
address	manages address-related info
–format=sexp	returns results ready for Emacs
–format-version=5	locks the version to avoid surprises
–sort=oldest-first	reverses the order so the oldest appears first

This command returns a list with the senders of the thread and their addresses. We reverse the order so the oldest sender — the thread creator — comes first.

((:name "Name1" :address "name1@company.com" :name-addr "Name1 <name1@company.com>")
(:name "Name2" :address "name2@company.com" :name-addr "Name2 <name2@company.com>")
(:name "Name3" :address "name3@company.com" :name-addr "Name3 <name3@company.com>"))

Once you get this, writing the function is easy. It opens a new frame with a search query by sender in two flavors:

A query using only the sender like from:add@company.com with the notmuch-search function
A query that adds the sender to the current query using the notmuch-search-filter function

(defun notmuch-search-find-sender (add_filter)
  "Open a new search filter by sender."
  (interactive "P")
  (let* ((args (list "address"
                     "--format=sexp"
                     "--format-version=5"
                     "--output=sender"
                     ;; "--deduplicate=address"
                     "--sort=oldest-first"
                     (format "%s" (notmuch-search-find-thread-id))))
         (address (apply #'notmuch-call-notmuch-sexp args))
         (filter-string (format "from:%s" (plist-get (car address) :address))) )
    (if add_filter
        (notmuch-search-filter filter-string)
      (notmuch-search filter-string))))

I’ve assigned the function to the f key for “from”, and if you call it with C-u f it runs the filter adding it to the current one.

Thread tag management

This topic is more complex because it depends on how you sync and tag your emails. In my case, I have a bash script that handles this whole process.

Thread tagging needs to be done right after integrating new mail with notmuch new. At that point, the new messages are tagged with new and from there we can pull the threads.

You need to consider which tags you don’t want to touch. For example, if one message in the thread is tagged with draft, you don’t want that to be replicated across the thread.

Since bash isn’t really my strong point, I have to admit I asked the AI to generate the first version, and from there I adapted it to my setup and needs. Here it is:

# Title for logging
APP="TagT"

# Counters for logging
threads_new=0
threads_modified=0
modified_threads=()

# Protected tags
PROTECTED_FIXED=(
    inbox
    unread
    trashed
    attachment
    gestmail
    gmail
    fastmail
    new
    flag
    replied
    forwarded
    draft
    sent
)

is_protected() {
    local tag="$1"

    # Any tag that begin by "zr*". This is a special tag, a number prfix by "zr"
    case "$tag" in
                zr*) return 0 ;;
    esac

    for p in "${PROTECTED_FIXED[@]}"; do
                [[ "$tag" == "$p" ]] && return 0
    done

    return 1
}

# Get all threads with the new tag. Thats why we have to do it just
# after the notmuch new
threads=$(notmuch search --output=threads --format=text tag:new)

for thr in $threads; do
        # +1 thread treated
    ((threads_new++))

    # Threads with only 1 message. New thread, nothing to do.
    if (( $(notmuch count "${thr}") <= 1 )); then
                continue
    fi

    # array for tags
    declare -A union_tags=()

    # Loop for all messages in the thread
    while read -r msgid; do
                [[ -z "$msgid" ]] && continue

                # tags from the messages
                for tag in $(notmuch search --output=tags "${msgid}"); do
                        if ! is_protected "$tag"; then
                                union_tags["$tag"]=1
                        fi
                done

    done < <(notmuch search --output=messages --format=text "${thr}")

    # Create the arguments to add the tags with notmuch
    add_args=()
    for tag in "${!union_tags[@]}"; do
                add_args+=( "+${tag}" )
    done

    # Apply the new tags to the thread. But only the good ones.
    if ((${#add_args[@]})); then
                notmuch tag "${add_args[@]}" "${thr}"

                # +1 thread modified
                ((threads_modified++))

                # keep thread for logging
                modified_threads+=("${thr}")
    fi

    # Clean the tags array
    unset union_tags
done

# Logging info.
logger -p user.info -t "${APP}" "Threads summary ================="
logger -p user.info -t "${APP}" "   New Threads: ${threads_new}"
if [ ${threads_modified} -gt 1 ]; then
        logger -p user.info -t "${APP}" "   Modified Threads: ${threads_modified=}"
        for thr in "${modified_threads}"; do
                logger -p user.info -t "${APP}" "      Thread: ${thr}"
        done
fi

I tried to comment the most interesting parts, but the final summary is that from the messages tagged as new we extract the threads:

notmuch search --output=threads --format=text tag:new

To get the tags from the messages, we use the output of search with the --output=tags parameter, which returns the tags of the message:

notmuch search --output=tags "${msgid}"

From these, we collect the tags (excluding the protected ones), store them in ${add_args[@]}, and apply them globally to the thread ${thr}:

notmuch tag "${add_args[@]}" "${thr}"

Another interesting bit is the loop through messages in the thread ${thr}:

notmuch search --output=messages --format=text "${thr}"

This gives us all the messages in the thread, which we iterate over to extract the tags.

Then I created these functions in Emacs to fix old threads. They work in both notmuch-search-mode and notmuch-tree-mode. In both modes the thread is retrieved differently—it's easier in tree-mode because it's already stored in the notmuch-tree-basic-query variable.

There are still some potential bugs to fix, etc., but I'll handle that later. For today, I think this is good enough. ☺️

(defun cnr/notmuch-search-homo-tread-tags ()
  "Homogenize tags in the selected thread."
  (interactive)
  (cond
   ((eq major-mode 'notmuch-search-mode)
    (let ((tags (cnr/notmuch-get-messages-from-thread (notmuch-search-find-thread-id))))
      (message "TAGS search: %s" (append notmuch-message-cleaned-tags tags))
      (notmuch-search-tag (append notmuch-message-cleaned-tags tags))))
     ((eq major-mode 'notmuch-tree-mode)
      (let ((tags (cnr/notmuch-get-messages-from-thread notmuch-tree-basic-query)))
      (message "TAGS tree: %s" (append notmuch-message-cleaned-tags tags))
      (notmuch-tree-tag-thread (append notmuch-message-cleaned-tags tags))))
  ))

(defun cnr/notmuch-get-messages-from-thread (id)
  (let* ((args '("search" "--format=sexp"))
         (protected-tags '("inbox" "unread" "trashed" "attachment" "gestmail" "gmail" "fastmail" "new" "flag" "replied" "forwarded" "draft" "sent"))
         (messages (apply #'notmuch-call-notmuch-sexp (append args (list "--output=messages" id))))
         (tag-list '()))
    (setq tag-list (delete-dups (mapcan (lambda (msg-id)  (apply #'notmuch-call-notmuch-sexp (append args (list "--output=tags" (format "id:%s" msg-id))))) messages)))
    (mapcar (lambda (x) (concat "+" x)) (seq-filter
     (lambda (x) (not (string-prefix-p "zr" x)))
      (seq-difference tag-list protected-tags)))))

Alright, that’s it for now! Hopefully, some of this helps you out someday. If you’ve got questions, don’t be shy — just reach out.

Emacs is the one editor to rule them all. 💪🏻