Protesilaos Stavrou

⚠️ IMPORTANT 2022-11-15: Please help me find a job and move countries.

Change Log of Denote

Simple notes for Emacs with an efficient file-naming scheme

This document contains the release notes for each tagged commit on the project's main git repository: https://git.sr.ht/~protesilaos/denote.

The newest release is at the top. For further details, please consult the manual: https://protesilaos.com/emacs/denote.

Version 1.1.0 on 2022-10-20

New commands or refinements to common use-cases

  • The denote-link-add-missing-links is a companion to what we already provide to produce a list of links to Denote files matching a regular expression (the denote-link-add-links). This new command adds links that are not already present in the current file. So if you have a metanote that references, say, your journal entries but have not updated it in a month, you can revisit the metanote, invoke denote-link-add-missing-links, and then type the search terms (e.g. _journal) to include what remains.

    Thanks to Elias Storms for the initial contribution, which was done in pull request 108 on the GitHub mirror: https://github.com/protesilaos/denote/pull/108.

    Elias has assigned copyright to the Free Software Foundation. It is required for changes that exceed 15 lines in total.

  • The denote-link-find-backlink provides a minibuffer interface that shows all backlinks to the current note. It complements the existing denote-link-backlinks command (which also has the alias denote-link-show-backlinks-buffer). Each command has its own niche: the minibuffer lets the user leverage powerful pattern matching styles, such as those provided by the orderless package, while the bespoke buffer provides an easy overview of what links to the current note.

    Thanks to Elias Storms for the original patch: https://lists.sr.ht/~protesilaos/denote/%3Cm2fsg6o2t6.fsf%40MBA21.fritz.box%3E#%3Cm2pmfam7yi.fsf@MBA21.fritz.box%3E.

  • The denote-keywords-add and denote-keywords-remove are two commands that interactively operate on the current note's front matter to add or remove keywords. They use the familiar keywords' prompt which means, among others, that they can read more than one keyword at a time. To specify multiple keywords, separate each input with a comma (or whatever the value of crm-separator is, which should be a comma unless something out-of-the-ordinary is in force).

    Thanks to Elias Storms for the original patch, which was done as part of a discussion on the mailing list and then iterated on: https://lists.sr.ht/~protesilaos/denote/%3Cm24jwvpbt2.fsf%40MBA21.fritz.box%3E#%3Cm28rlik0tc.fsf@MBA21.fritz.box%3E.

  • The denote-link command will now recognise an active region and use its text as the description of the inserted link. The default behaviour is to use the file's title from its front matter or file name. Thanks to Charanjit Singh for the original contribution, which was done as part of pull request 109 on the GitHub mirror: https://github.com/protesilaos/denote/pull/109. A subsequent tweak was implemented in pull request 110, following a discussion with me: https://github.com/protesilaos/denote/pull/110.

    Charanjit's contribution is below the ~15 line threshold and thus does not require copyright assignment to the Free Software Foundation.

  • The renaming operations are now aware of the underlying version control system and will use the appropriate command when a VCS is available. In practice, renaming a file under, say, Git will register it as a "rename" instead of two separate actions of deletion and addition.

    Thanks to Florian for the patch. It was discussed on the mailing list and then underwent some changes: https://lists.sr.ht/~protesilaos/denote/%3C166547153518.8.941129310186454444.68125516@aboulafia.org%3E.

  • The denote-rename-file-using-front-matter no longer fails to carry out its intended task when the front matter has no keywords. If no keywords are available, this is interpreted as a request to remove the KEYWORDS component of the file name. This was always technically possible and could be achieved with various permutations of the user option denote-prompts (as explained in its doc string or the manual). Denote only needs an identifier in the file name to establish unique links (although I strongly encourage you to stick to the standard file-naming scheme as it is informative, reliable, and can work even if you access your data without Emacs).

For more advanced use-cases

  • The variable denote-file-types has been tweaked to respond directly to changes in its value done with setq. Thanks to Noboru Ota for the patch: https://lists.sr.ht/~protesilaos/denote/%3C86k05gsqsg.fsf%40nobiot.com%3E.

    Noboru has assigned copyright to the Free Software Foundation.

  • The :front-matter property of the denote-file-types now accepts a nil value. Denote could always work without front matter, but this was not implemented flexibly in the denote-file-types. Thanks to Noboru Ota (nobiot) for pointing this out on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C86k05gsqsg.fsf%40nobiot.com%3E.
  • The denote-file-prompt function now reads an optional INITIAL-TEXT argument. This is a string that prepopulates the minibuffer. It is useful for custom commands the user may have where, for example, there is a need to automatically filter to entries matching _journal. Thanks to Alan Schmitt for suggesting the idea: https://lists.sr.ht/~protesilaos/denote/%3C87pmf676n1.fsf@m4x.org%3E.
  • The denote-rename-file-using-front-matter accepts an optional AUTO-CONFIRM argument. It can either be passed interactively or via Lisp. The doc string (or the manual) explains the details. Thanks to Elias Storms for the initial patch: https://lists.sr.ht/~protesilaos/denote/%3Cm2a667aeku.fsf%40gmail.com%3E.
  • The denote-prompt-for-date-return-id function uses the familiar denote-date-prompt and returns the appropriate identifier. It is used internally by some of our function, but we also provide it for anyone who wants to write their own custom code.
  • The denote-retrieve-or-create-file-identifier function reads and option DATE argument to its mandatory FILE argument. If FILE does not have an identifier and optional DATE is non-nil, the function invokes the denote-prompt-for-date-return-id, as mentioned above.

  • The denote-rename-file command accepts an optional DATE argument. It functionally does what is described right above, with the exception that this is for an interactive function (a "command"). Read the detailed doc string or the manual for everything that pertains to this powerful command.

    Thanks to Florian for suggesting the idea on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C166521684647.7.5483179875879361874.67576870%40aboulafia.org%3E.

  • The denote-directory-text-only-files function filters the denote-directory-files to only return a list of text files. This leaves out, say, mp3 files. The function is used internally, though it may also prove useful in custom user code.

Miscellaneous refinements

  • Implemented a revert-buffer-function for the backlinks' buffer, which is produced by the command denote-link-backlinks. This revert function is what the g key invokes with the default key bindings (the command is revert-buffer). It produces the buffer anew, updating the list of backlinks accordingly.

  • Documented how to speed up the creation of the backlinks' buffer. As this depends on the built-in xref library, the change is done by specifying the value of the user option xref-search-program in Emacs 28 or higher. For example: 

      (setq xref-search-program 'ripgrep)

    For something more elaborate: 

      ;; Prefer ripgrep, then ugrep, and fall back to regular grep.
  (setq xref-search-program
        (cond
         ((or (executable-find "ripgrep")
              (executable-find "rg"))
          'ripgrep)
         ((executable-find "ugrep")
          'ugrep)
         (t
          'grep)))
  • Removed some minor duplication of effort in how the buttonisation of links is done (what makes them clickable).
  • Made refinements to the definition of functions such as denote-link-add-links. There should be no noticeable change for users, though this shows we care about code quality.
  • With Eshel Yaron, we tried to remove the empty indices for functions and variables from the HTML version of the manual. These indices are useful in the Info version, which can be accessed directly from Emacs when the denote package is installed (for example, evaluate (info "(denote) Top")), but they do not work with HTML. Alas, what we tried to do did not work. Maybe Org has a way to control what is exported where. We shall see. At any rate, thanks to Eshel for the effort: https://lists.sr.ht/~protesilaos/denote/patches/36028.
  • All code that integrates the denote: custom hyperlink type with Org's link facility is now assigned autoload cookies. These are done to ensure that denote is loaded and is available in cases where Org needs to access a denote: link at some early stage (e.g. at startup before using Denote). Thanks to Sven Seebeck for reporting the problem: https://lists.sr.ht/~protesilaos/denote/%3C87r0zovwix.fsf%40svenseebeck.me%3E. Although Sven could not reproduce a bug reliably, I believe this prevents such an eventuality.
  • Expanded or otherwise updated the manual to account for all of the above, where appropriate.

Version 1.0.0 on 2022-09-30

This is the first major release of Denote. A part of the changes documented herein is for advanced users or developers who wish to extend Denote with their custom code. Though we first cover what applies to everyone.

Changes for all users

  • The custom Org hyperlink type of denote: can be visited from outside the denote-directory. We now provide the necessary glue code that Org needs to store these denote: links. Storing them can be done with an org-capture template or via the command org-store-link. Use this to, for example, capture a TODO that references one of your notes.

    denote: links work for as long as the referenced file is somewhere in the denote-directory or one of its subdirectories.

    Thanks to Marc Fargas for the contribution. Marc did not need to assign copyright to the Free Software Foundation, as the patch was within the ~15 line limit that is permissible.

    The contribution was discussed on the mailing list: https://lists.sr.ht/~protesilaos/denote/patches/35137. A prior exchange took place in issue 104 over at the GitHub mirror: https://github.com/protesilaos/denote/issues/104.

    Some further tweaks were made to the relevant function. Thanks to Elias Storms for reporting on the mailing list a bug which revealed a regression I introduced to the Org link storing mechanism: https://lists.sr.ht/~protesilaos/denote/%3C15D55F4B-64D1-4083-AD5E-B5BACA8F1909%40ap.be%3E.

  • Following from above, the command denote-link-find-file finds files reliably, regardless of where the link is stored. All it needs is for the target file to be inside the denote-directory.

    I discovered this while exchanging views with Marc Fargas regarding the aforementioned patch: https://lists.sr.ht/~protesilaos/denote/patches/35137.

  • The command denote-link-buttonize-buffer, which "buttonizes" denote: links in plain text and Markdown files, now performs its task regardless of where the current file is stored. Those links work for as long as the file they reference is somewhere inside the denote-directory.

  • The commands denote-link-after-creating, denote-link-or-create provide a convenience for users who need to create link to notes that may not exist yet. The idea is that one is expounding on a given topic and wants to create a link to a relevant issue. They are not sure if they have written anything about it yet, so they invoke the relevant command. Consult their doc strings or read the manual: https://protesilaos.com/emacs/denote#h:9e41e7df-2aac-4835-94c5-659b6111e6de.

    Thanks to user sienic for suggesting the idea and for testing the prototypes. And thanks to Juanjo Presa for participating in the discussion to share the view that this functionality should be part of denote.el. This happened in issue 96 over at the GitHub mirror: https://github.com/protesilaos/denote/issues/96.

  • The command denote-open-or-create offers the convenience of visiting a file, if it exists, else prompting for its creation. Thanks to Alan Schmitt for the contribution. The patch was sent on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C87fsgvddny.fsf%40protesilaos.com%3E. It is within the limit of what is allowed without assigning copyright to the Free Software Foundation, though Alan has done the relevant paperwork.
  • The manual expands on two sections: (1) Variants of denote-open-or-create, (2) Variants of denote-link-or-create. They show how one can use the above "do or create" commands with different permutations of the Denote prompts for new note creation.

  • The manual includes a section titled "Create a note with the region's contents". Quote:

    Sometimes it makes sense to gather notes in a single file and later review it to make multiple notes out of it. With the following code, the user marks a region and then invokes the command my-denote-create-new-note-from-region: it prompts for a title and keywords and then uses the region's contents to fill in the newly created note.

    This is not part of denote.el, though we provide it in the manual for users that may need it. Thanks to sundar bp for suggesting the idea. This was done via a private channel and the information is shared with permission.

  • The manual has another entry titled "Split an Org subtree into its own note", which is similar to the above idea of using the region's contents but has some extra niceties provided by Org. Quote:

    With Org files in particular, it is common to have nested headings which could be split off into their own standalone notes. In Org parlance an entry with all its subheadings is a "subtree". With the following code, the user places the point inside the heading they want to split off and invokes the command my-denote-split-org-subtree. It will create a note using the heading's text and tags for the new file. The contents of the subtree become the contents of the new note and are removed from the old one.

    Thanks to Sven Seebeck for suggesting the idea and for testing my prototypes. This information is shared with permission, as it was provided via a private channel.

  • The manual describes how a user can leverage the built-in dired-virtual-mode to perform arbitrary sorting of their list of notes. It also includes code for Eshell to quickly "export" a command's output into a dedicated buffer (which can then be used to derive a "virtual" Dired). Thanks to Yi Liu for asking the question that inspired this entry: https://lists.sr.ht/~protesilaos/denote/%3C1C75FF01-EC76-49DF-9AEB-ED718A2795FF@gmail.com%3E.
  • The denote-faces-broken-link has been removed. It was used for Org links. The idea was to apply a different style if the link was broken. However, the way fontification works means that there may be a performance penalty as Org tries to check again and again if the link is broken or note. As denote: links are robust (unless the user tries to break them), this penalty is unacceptable. Thanks to Peter Prevos for reporting the issue and discussing it with me on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C87k05umyyo.fsf%40prevos.net%3E.
  • The "denote" group in Custom UI buffers now provides a link to the Info manual that is shipped with the package. To read the manual, evaluate (info "(denote) Top"). Else visit the official web page: https://protesilaos.com/emacs/denote.
  • Fixed a case where an internal check for a note would throw an error if the buffer was not visiting a file. Thanks to Hilde Rhyne was the patch: it is below the ~15 line threshold and thus does not require copyright assignment to the Free Software Foundation. The issue was discussed on the mailing list and was pushed to users as version 0.6.1: https://lists.sr.ht/~protesilaos/denote/%3Cm035d7nq22.fsf%40disroot.org%3E.
  • When linking to a file that has no front matter, Denote tries to use the TITLE component of the file name (per our file-naming scheme) as the link's descriptive text. We now make this look a bit better, by capitalising only the first letter while dehyphenating the text, converting this-is-a-test to This is a test. Before, we would capitalise all words. Thanks to Clemens Radermacher for the patch. It was sent via a private channel. Clemens has assigned copyright to the Free Software Foundation.

Changes for developers or advanced users

Lots of functions and variables which once were for "private" use (the presence of double hyphens in the symbol) are now made public. Concretely this means that they no longer have double hyphens in their name and we pledge to support them henceforth. "Support" means that we (i) consider them stable, (ii) document them properly, (iii) will record any changes made to them such as in a change log, a blog post on my website, and via make-obsolete.

The manual provides a complete reference of what is on offer. The section is titled "For developers or advanced users": https://protesilaos.com/emacs/denote#h:c916d8c5-540a-409f-b780-6ccbd90e088e.

Normally, we do not support private forms and can delete/modify them without notice. However, I decided to write obsoletion aliases for all forms I made public or otherwise revised, in an effort not to break any existing custom code. The following table covers all obsolete symbols and their new counterparts. PLEASE UPDATE YOUR CODE as those aliases will be removed in the near future.

Index Old symbol New symbol
1 denote–id-format denote-id-format
2 denote–id-regexp denote-id-regexp
3 denote–title-regexp denote-title-regexp
4 denote–keywords-regexp denote-keywords-regexp
5 denote–punctuation-regexp denote-excluded-punctuation-regexp
6 denote-punctuation-excluded-extra-regexp denote-excluded-punctuation-extra-regexp
7 denote–sluggify denote-sluggify
8 denote–sluggify-and-join denote-sluggify-and-join
9 denote–sluggify-keywords denote-sluggify-keywords
10 denote–desluggify denote-desluggify
11 denote–only-note-p denote-file-is-note-p
12 denote–file-has-identifier-p denote-file-has-identifier-p
13 denote–file-supported-extension-p denote-file-has-supported-extension-p
14 denote–writable-and-supported-p denote-file-is-writable-and-supported-p
15 denote–file-name-relative-to-denote-directory denote-get-file-name-relative-to-denote-directory
16 denote-link–id-from-string denote-extract-id-from-string
17 denote–directory-files denote-directory-files
18 denote–subdirs denote-directory-subdirectories
19 denote–get-note-path-by-id denote-get-path-by-id
20 denote–directory-files-matching-regexp denote-directory-files-matching-regexp
21 denote–retrieve-read-file-prompt denote-file-prompt
22 denote–extract-keywords-from-path denote-extract-keywords-from-path
23 denote–keywords-prompt denote-keywords-prompt
24 denote–retrieve-filename-identifier denote-retrieve-filename-identifier
25 denote–file-name-id denote-retrieve-or-create-file-identifier
26 denote–retrieve-filename-title denote-retrieve-filename-title
27 denote–retrieve-title-value denote-retrieve-title-value
28 denote–retrieve-title-line denote-retrieve-title-line
29 denote–retrieve-keywords-value denote-retrieve-keywords-value
30 denote–retrieve-keywords-line denote-retrieve-keywords-line
31 denote–format-file denote-format-file-name
32 denote–barf-duplicate-id denote-barf-duplicate-id
33 denote–title-prompt denote-title-prompt
34 denote–file-type-prompt denote-file-type-prompt
35 denote–date-prompt denote-date-prompt
36 denote–subdirs-prompt denote-subdirectory-prompt
37 denote–template-prompt denote-template-prompt
38 denote–filetype-heuristics denote-filetype-heuristics
39 denote–rename-file denote-rename-file-and-buffer
40 denote–rename-file-prompt denote-rename-file-prompt

If you are writing code that extends Denote and feel that something is either missing or has remained private, please contact us on the mailing list, the GitHub/GitLab mirror, or send me an email directly. I always respond in a timely fashion.

Open to everyone

The most common feedback I get about Denote is that its documentation is good. As you can tell from these change logs, the plan is to continue on this path.

Please note that the communication channels for Denote (mailing list, mirrors, my personal email) are open to users of all levels. Do not hesitate to contact us/me.

Thanks again to everyone for their contributions, direct or indirect, either in the form of code or the discussion of ideas. Quoting from the "Acknowledgements" section of the manual (all my packages have such a section):

Denote is meant to be a collective effort. Every bit of help matters.

Author/maintainer
Protesilaos Stavrou.
Contributions to code or the manual
Abin Simon, Alan Schmitt, Benjamin Kästner, Clemens Radermacher, Colin McLear, Damien Cassou, Eshel Yaron, Hilde Rhyne, Jack Baty, Jean-Philippe Gagné Guay, Jürgen Hötzel, Kaushal Modi, Kyle Meyer, Marc Fargas, Peter Prevos, Philip Kaludercic, Quiliro Ordóñez, Stefan Monnier.
Ideas and/or user feedback
Abin Simon, Alan Schmitt, Alfredo Borrás, Benjamin Kästner, Colin McLear, Damien Cassou, Elias Storms, Frank Ehmsen, Hanspeter Gisler, Jack Baty, Juanjo Presa, Kaushal Modi, M. Hadi Timachi, Paul van Gelder, Peter Prevos, Shreyas Ragavan, Summer Emacs, Sven Seebeck, Taoufik, Yi Liu, Ypot, atanasj, hpgisler, pRot0ta1p, sienic, sundar bp.

Special thanks to Peter Povinec who helped refine the file-naming scheme, which is the cornerstone of this project.

Special thanks to Jean-Philippe Gagné Guay for the numerous contributions to the code base.

Version 0.6.0 on 2022-08-31

Denote is in a stable state. I consider it feature-complete, without prejudice to possible refinements to its existing feature set. The next version shall be 1.0.0.

User-facing changes

  • The Denote linking facility can now link to any file that has the Denote file-naming scheme. Before, we limited this feature to what we consider "note" files, else the supported plain text formats (per denote-file-type). Thanks to Peter Prevos for the discussion on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C87fsi1m5ze.fsf%40prevos.net%3E.
  • Date prompts may optionally use the familiar Org date-selection mechanism that leverages the calendar. This feature is subject to the user option denote-date-prompt-use-org-read-date. A date prompt is used by the denote-date command or, optionally, by the denote command when the user option denote-prompts is configured accordingly. The manual elaborates on the specificities. Thanks to Jean-Philippe Gagné Guay for the contribution in pull request 97 at the GitHub mirror: https://github.com/protesilaos/denote/pull/97.

  • Leading empty spaces at the denote TITLE prompt no longer produce hyphens: they are simply ignored to keep file names consistent. Thanks to Peter Prevos for the contribution in pull request 99 at the GitHub mirror: https://github.com/protesilaos/denote/pull/99.

    [ Peter has started the process for copyright assignment to the Free Software Foundation, though the total contributions are still within the permitted boundaries. ]

  • When linking to files that have no front matter, the link's anchor text (the human-readable part) is derived from the file name TITLE component. We apply a de-hyphenation and capitalisation of its constituent words. This is not always perfect, but it is better than something like this-is-the-title. Thanks to Peter Prevos for the original idea in pull request 93 at the GitHub mirror: https://github.com/protesilaos/denote/pull/93.

  • The active region is now used as the default value of the denote command TITLE prompt. The idea behind this Do-What-I-Mean-flavoured patch is to be able to take a note about a subject that appears in a buffer by simply marking it before invoking the denote command.

    Thanks to Eshel Yaron for the patch: https://lists.sr.ht/~protesilaos/denote/patches/34870. It is below the ~15 line threshold that thus requires no copyright assignment to the Free Software Foundation.

  • The denote-rename-file-using-front-matter command now offers to save the buffer if appropriate. In the past, it would simply produce an error asking the user to save the buffer. Thanks to Peter Prevos for the contribution in pull request 103 at he GitHub mirror: https://github.com/protesilaos/denote/pull/103.

  • Fixed the text of the confirmation prompt in the command denote-migrate-old-markdown-yaml-tags. Thanks to Abin Simon for the patch: https://lists.sr.ht/~protesilaos/denote/patches/34632.

    This patchset also fixes (i) how a tag is identified for the purposes of migrating old to new front matter, (ii) the regular expression for Org front matter keywords

    [ The total changes are below the ~15 line threshold and thus do not require copyright assignment to the Free Software Foundation. ]

  • Fixed a bug that prevented the creation of new notes. Thanks to Juergen Hoetzel for the contribution in pull request 84 at the GitHub mirror: https://github.com/protesilaos/denote/pull/84. This was done immediately after the release of version 0.5.0 on 2022-08-10 and was provided to users as version 0.5.1

    [ The change is below the ~15 line threshold. ]

Internal refinements

These make the code simpler and more predictable. As the individual changes are not user-facing, I invite interested parties to consult the Git log. Special thanks to Jean-Philippe Gagné Guay for the multiple contributions (and relevant discussions) over at the GitHub mirror:

[ Jean-Philippe has assigned copyright to the Free Software Foundation. It is required for non-trivial changes. ]

For advanced users

The variable denote-file-types is an alist of plists which substantiates the supported file types (per the user option denote-file-type). Properties pertain to the formatting of front matter and the retrieval of relevant values. The doc string of denote-file-types explains the details, while the default value uses the ancillary functions we define. Thanks to Jean-Philippe Gagné Guay for the relevant contributions in pull request 89 at the GitHub mirror: https://github.com/protesilaos/denote/pull/89.

Version 0.5.0 on 2022-08-10

The general theme of this release is to refine what we already offer. As I explained in some discussions, Denote is feature-complete. We can always improve the code or add some ancillary function/command/variable, though all the main ideas have already been implemented. Additional functionality can be provided by other packages: I remain at the disposal of anyone willing to write such a package.

The present release covers more than 150 commits since version 0.4.0 on 2022-07-25.

All release notes: https://protesilaos.com/emacs/denote-changelog.

Templates for new notes

We now provide the denote-templates user option. A "template" is arbitrary text that Denote will add to a newly created note right below the front matter.

Templates are expressed as a (KEY . STRING) association.

  • The KEY is the name which identifies the template. It is an arbitrary symbol, such as report, memo, statement.
  • The STRING is ordinary text that Denote will insert as-is. It can contain newline characters to add spacing. The manual of Denote contains examples on how to use the concat function, beside writing a generic string: https://protesilaos.com/emacs/denote#h:f635a490-d29e-4608-9372-7bd13b34d56c.

The user can choose a template either by invoking the new command denote-template or by changing the user option denote-prompts to always prompt for a template when calling the denote command.

Thanks to Jean-Philippe Gagné Guay for refinements to this facility. Done in pull request 77 on the GitHub mirror: https://github.com/protesilaos/denote/pull/77.

[ Jean-Philippe has assigned copyright to the Free Software Foundation. ]

Revised format for Org #+filetags entry

Denote used to format tags in Org files by separating them with two spaces: 

#+filetags:  tag1  tag2

While this worked for some obvious use-cases, it is not supported by Org. The Org documentation stipulates that tags be separated by the colon sign. The above would then be written thus: 

#+filetags:  :tag1:tag2:

Denote now conforms with Org's specifications. To help users update their existing notes, we provide the denote-migrate-old-org-filetags command. It will perform the conversion in all Org files that had the old notation. As with all Denote operations that rewrite file contents, it DOES NOT SAVE BUFFERS. The user is expected to review the changes, such as by using diff-buffer-with-file. Multiple buffers can be saved with save-some-buffers (check its doc string).

This command is provided for the convenience of the user. It shall be deprecated and eventually removed from future versions of Denote.

If you need help with any of this, please do not hesitate to contact me either in private or in one of Denote's official channels (mailing list, GitHub/GitLab mirror).

Thanks to Alan Schmitt for bringing this matter to my attention: https://lists.sr.ht/~protesilaos/denote/%3C871qu0jw5l.fsf%40protesilaos.com%3E. Also thanks to Jean-Philippe Gagné Guay for commenting on it as it helped me decide to include the command in denote.el: https://github.com/protesilaos/denote/pull/83#issuecomment-1210167133.

Revised format for Markdown+YAML tags: entry

This is the same idea as above. Before, we were making the mistake of using incorrect YAML notation: 

tags:  tag1  tag2

Now we do: 

tags:  ["tag1", "tag2"]

This is how the TOML variant always worked.

For the user's convenience, we provide a command to migrate from the old to the new syntax: denote-migrate-old-markdown-yaml-tags.

Changes to file renaming and front matter rewriting

Denote adds "front matter" to newly created notes which includes data such as the title and keywords/tags of the document. Strictly speaking, the front matter is not required by Denote. It is provided for the user's convenience, such as for readability or if they want to use the note with other programs (e.g. Org export, a blog with Hugo/Jekyll, …).

Denote provides commands which help the user rename their notes, by changing the file name's TITLE and/or KEYWORDS components (per Denote's file-naming scheme). These commands also operate on the front matter to keep the data between file name and file contents in sync (again, for the user's convenience).

For this release we have consolidated and refined our offerings in order to improve their ergonomics. All changes are the result of fruitful discussions on the mailing list and the issue tracker of the GitHub mirror:

Thanks to (A-Z) Hanspeter Gisler, Jean-Philippe Gagné Guay, and Peter Prevos for their participation.

Also thanks to Jean-Philippe Gagné Guay for relevant code contributions (please consult the Git log for the minutiae):

Renaming a single file

The commands denote-dired-rename-file-and-add-front-matter and denote-dired-rename-file are deprecated and superseded by the new denote-rename-file. Please update any key bindings in your setup.

The difference between the old commands and the new denote-rename-file is that the latter will now insert front matter to supported file types (per denote-file-type) if they have none. This basically means that, e.g., renaming a generic Org/Markdown/Plain text file with denote-rename-file will update its file name to comply with Denote's file-naming scheme and also add the appropriate front matter (it "converts" it to a Denote note). If front matter exists, this command will rewrite it to reflect the changes to the file name's TITLE and/or KEYWORDS.

Consult the manual for the details: https://protesilaos.com/emacs/denote#h:7cc9e000-806a-48da-945c-711bbc7426b0.

Or, if the new version of the GNU ELPA package is installed, evaluate: 

(info "(denote) Rename a single file")

The user option denote-dired-rename-expert is obsolete. Denote always asks for confirmation when renaming a single file. This is because the user can rely on batch-renaming commands which ask for confirmation only once per batch.

Renaming multiple files at once

The command denote-dired-rename-marked-files-and-add-front-matter is deprecated and its functionality is absorbed by the existing denote-dired-rename-marked-files command. The deprecated command was used to insert front matter to supported file types (per denote-file-type) that had none. We now handle this internally, thus streamlining the experience for the user.

Refer to the manual for the details: https://protesilaos.com/emacs/denote#h:1b6b2c78-42f0-45b8-9ef0-6de21a8b2cde

Assuming the latest Info manual is installed, evaluate: 

(info "(denote) Rename multiple files at once")

Renaming a single file based on its front matter

Introduced the denote-rename-file-using-front-matter command. This is new functionality we provide which uses the front matter as input to perform a rename of the file. The aforementioned offerings prompt for input via the minibuffer and propagate the changes firstly to the file name and subsequently to the front matter. Whereas with the command denote-rename-file-using-front-matter, the user can edit the front matter manually and then invoke the command to pass the changes to the file name, subject to a confirmation. Relevant entries are the title and tags/filetags (depending on the file type). The date and the identifier are not pertinent. Identifiers in file names are NEVER rewritten by Denote.

Consult the manual: https://protesilaos.com/emacs/denote#h:3ab08ff4-81fa-4d24-99cb-79f97c13a373.

With the latest package, evaluate: 

(info "(denote) Rename a single file based on its front matter")

Renaming multiple files based on their front matter

The command denote-dired-rename-marked-files-using-front-matter completes the set of features we provide for syncing between file name and front matter. It applies to all marked files in a Dired buffer.

Read the manual to understand how the command works and what it does exactly: https://protesilaos.com/emacs/denote#h:ea5673cd-e6ca-4c42-a066-07dc6c9d57f8.

Or evaluate: 

(info "(denote) Rename multiple files based on their front matter")

Add missing front matter on demand

Sometimes the user may have incomplete front matter, perhaps due to a mistake that was saved on disk. The command denote-add-front-matter appends a new front matter block to the current note.

Read: https://protesilaos.com/emacs/denote#h:54b48277-e0e5-4188-ad54-ef3db3b7e772

Or evaluate: 

(info "(denote) Regenerate front matter")

Faces for Denote links

We provide the denote-faces-link and the denote-faces-broken-link. The latter is only relevant for Org, as Emacs' standard button mechanism does not have a way to apply a face dynamically.

This is a change for themes/tinkerers who need to differentiate denote: links from other links. Otherwise, the presentation is the same as before.

Thanks to Peter Prevos for asking about it on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C03618bb20d3eaba78c32cd0cb63bfc71%40prevos.net%3E.

Use of XDG path in denote-directory

The default value of the denote-directory user option used to be ~/Documents/notes (subject to some conversion via Elisp). Denote now conforms with the freedesktop.org specifications by using the XDG directory for DOCUMENTS instead of ~/Documents: https://www.freedesktop.org/wiki/Software/xdg-user-dirs/.

Users who already bind the denote-directory are not affected by this change. Same for those who do not tinker with XDG environment variables and/or do not use some exotic setup.

Thanks to Philip Kaludercic for the patch: https://lists.sr.ht/~protesilaos/denote/patches/34561#%3C20220809115824.43089-1-philipk@posteo.net%3E

Bespoke major-mode for the backlinks' buffer

The backlinks' buffer now uses the denote-backlink-mode instead of the generic special-mode. The former derives from the latter. It binds keys to move between links with n (next) and p (previous). These are stored in the denote-backlink-mode-map (use M-x describe-mode (C-h m) in an unfamiliar buffer to learn more about it).

Thanks to Philip Kaludercic for the patch: https://lists.sr.ht/~protesilaos/denote/patches/34561#%3C20220809115824.43089-2-philipk@posteo.net%3E

Changes to the manual

  • Documented all of the aforementioned. Improved how information is presented and, generally, iterated on an already comprehensive document.

  • Introduced a node which explains how to tweak the front matter: https://protesilaos.com/emacs/denote#h:7f918854-5ed4-4139-821f-8ee9ba06ad15. Or evaluate: 

      (info "(denote) Change the front matter format")

  • Updated the reference to consult-notes. This is a package that uses the consult interface to provide access and search facilities for notes. It can integrate with Denote. Thanks to Colin McLear for the change in pull request 70 on the GitHub mirror: https://github.com/protesilaos/denote/pull/70.

    [ The change is below the ~15 line threshold and thus does not require copyright assignment to the Free Software Foundation. ]

Internal restructuring

  • All Denote code is consolidated in denote.el. We no longer maintain separate files like denote-link.el, denote-dired.el, etc. Users who had require calls to such libraries must remove them and only keep: 

      (require 'denote)

  • User options that have an entry in the manual will now provide a link to it via their Help buffer and/or the Custom UI. This is done by adding the :link attribute to their declaration.

    Furthermore, user options and faces now specify the version of Denote that last affected their value (e.g. denote-directory, which was mentioned above for the XDG spec, now informs the user that it changed for version 0.5.0).

    [ I learnt these by developing the modus-themes. ]

  • The variables denote-last-title, denote-last-keywords, denote-last-buffer, and denote-last-front-matter are all obsolete. These were used prior to version 0.1.0 to help with development but are now deemed surplus to requirements.
  • Lots of changes were made to private functions, variables, doc strings, and comments, in the interest of simplifying the code and/or ensuring consistency in how operations are carried out. Though everything is the same for the end-user.

Thanks to Jean-Philippe Gagné Guay for the numerous contributions on the GitHub mirror. They are important for Denote, though the user does not need to know what is happening internally (consult the Git log for the details):

Discussions

Encrypting Denote notes

Paul van Gelder asked about this on the mailing list. I provided guidelines on what can be done, though did not record anything in the manual: I prefer to elicit more feedback from users. The gist is that Emacs already has all the requisite functionality, though encryption per se is outside the scope of Denote: https://lists.sr.ht/~protesilaos/denote/%3C1123434736.64290.1658954014673%40kpc.webmail.kpnmail.nl%3E.

Denote's relevant internal mechanisms will recognise files ending in .gpg (e.g. for fontification in Dired).

Visualise usage of Denote keywords

Peter Prevos shared a proof-of-concept way to visualise keywords in the denote-directory and show usage statistics: https://lists.sr.ht/~protesilaos/denote/%3Ce9e5d6ae85984b51067b47f4d8e134fa%40prevos.net%3E.

We do not include this information in the manual, as we wait for the fully fledged code. Though do give it a try if you are interested and, perhaps, share your thoughts for Peter's consideration.

Conflict between denote-dired-mode and diredfl-mode

Hilde Rhyne shared a workaround they have to disable diredfl-mode in the buffers where denote-dired-mode is enabled. The conflict between the two is a known issue that is acknowledged in the manual: https://lists.sr.ht/~protesilaos/denote/%3Cm0tu6q6bg0.fsf%40disroot.org%3E.

I think we need a proper solution in the code we provide, so this workaround is not mentioned in the manual.

Why doesn't Denote provide a search facility?

There was a discussion started by Fourchaux, with the participation of basaran and Andre0991 on the GitHub mirror: https://github.com/protesilaos/denote/issues/71.

The gist of my answer is that Denote does not need to provide such a facility because notes are ordinary files: whatever the user already has for them should apply to Denote. If the user has nothing to search through files, they anyhow need something that works outside the confines of Denote: a denote-SEARCH command is not an adequate solution.

Emacs has numerous built-in commands, such as grep (lgrep and rgrep), project-find-regexp, find-grep-dired, ibuffer-do-occur, … Furthermore, there are lots of high quality packages that have their own wrappers or extensions for searching file contents, such as the ivy and helm completion frameworks, as well as consult (the commands consult-grep and consult-ripgrep), consult-notes, rg, deadgrep, deft, and probably plenty more that do not come to mind right now.

I strongly encourage the user to find a universal search solution to the problem of searching file contents.

Version 0.4.0 on 2022-07-25

  • Defined the denote-link-dired-marked-notes command. It lets the user produce a typographic list of links to the note files that are marked in Dired. The list is written at point. If there are multiple buffers which visit Denote notes, the command first prompts with minibuffer completion for one among them.

    In terms of workflow, denote-link-dired-marked-notes complements the denote-link-add-links command for those cases where it is easier to select files than write an elegant regular expression.

  • Implemented the denote-dired-rename-marked-files command. This provides a much-requested facility to perform the familiar renaming operation on a set of files. In particular:

    • the file's existing file name is retained and becomes the TITLE field, per Denote's file-naming scheme;
    • the TITLE is sluggified and downcased, per our conventions;
    • an identifier is prepended to the TITLE;
    • the file's extension is retained;
    • a prompt is asked once for the KEYWORDS field and the input is applied to all file names;
    • if the file is recognised as a Denote note, the command rewrites its front matter to include the new keywords. A confirmation to carry out this step is performed once at the outset. Note that the affected buffers are not saved. The user can thus check them to confirm that the new front matter does not cause any problems (e.g. with the command diff-buffer-with-file). Multiple buffers can be saved with save-some-buffers (read its doc string).

    Parts of denote-dired-rename-marked-files were added or refined over a series of commits. Consult the Git log for the minutia. Thanks to Jean-Philippe Gagné Guay for the relevant additions in pull requests 51 and 52 on the GitHub mirror:

    Jean-Philippe has assigned copyright to the Free Software Foundation.

  • Improved how the denote-dired-rename-file command rewrites front matter. Before, it would perform a replacement of the whole block, which had the adverse effect of overwriting custom front matter entries. Now, it only targets the lines which hold the title and keywords, leaving everything else intact. Thanks to Peter Prevos for reporting the problem and testing the solution to it in issue 60 on the GitHub mirror: https://github.com/protesilaos/denote/issues/60.

  • Introduced the denote-dired-rename-file-and-add-front-matter command that always prepends front matter to a file whose extension is among the supported ones (per the user option denote-file-type). This differs from the standard denote-dired-rename-file command which only rewrites the front matter's title and keywords if they exist.

    In practice, denote-dired-rename-file-and-add-front-matter empowers the user to convert a generic text file to a Denote note.

    This command was originally added by Jean-Philippe Gagné Guay in pull request 49 on the GitHub mirror and refined in subsequent commits: https://github.com/protesilaos/denote/pull/49. Also read issue 48 where this idea was originally discussed: https://github.com/protesilaos/denote/issues/48.

  • Added the denote-dired-rename-marked-files-and-add-front-matters command, which is like the denote-dired-rename-marked-files but adds front matter instead of rewriting existing one, just how the command denote-dired-rename-file-and-add-front-matter does it (both are mentioned above). Thanks to Jean-Philippe Gagné Guay for the refinements to it in pull request 53 on the GitHub mirror: https://github.com/protesilaos/denote/pull/53.
  • Wrote an interactive spec for denote-link-buttonize-buffer. It can now be invoked with M-x or a key binding, should the need arise. This function is normally called via a hook and takes effect in plain text as well as Markdown files.

  • Extended the fontification rules so that file names with non-ASCII characters are styled properly. This issue was brought up on the mailing list by Frank Ehmsen and was discussed with the participation of Peter Prevos: https://lists.sr.ht/~protesilaos/denote/%3C2273b3b1-344c-6c6e-3ab6-a227b6bc3721%40eh-is.de%3E.

    The same topic was raised at the same time on the GitHub mirror by user hpgisler in issue 61: https://github.com/protesilaos/denote/issues/61.

    After some discussion, we agreed on the right approach, which was formalised by Peter Prevos as pull request 64 on the GitHub mirror: https://github.com/protesilaos/denote/pull/64. The change is below the ~15 line threshold and thus does not require copyright assignment to the Free Software Foundation.

  • Made the registration of the denote: custom Org hyperlink type conditional on the availability of the org feature. In other words, those who do not use Org will not be loading this part of the code. Thanks to Abin Simon for reporting the problem and for showing how Elfeed handles this case. This was done in issue 47 on the GitHub mirror: https://github.com/protesilaos/denote/issues/47.
  • Ensured that duplicate keywords are not produced by the relevant prompt. Thanks to user Taoufik for the contribution in pull request 50 on the GitHub mirror: https://github.com/protesilaos/denote/pull/50. The change is below the ~15 line threshold and thus does not require copyright assignment to the Free Software Foundation.
  • Fixed a typo in the reference to the crm-separator in the manual. David Wilson (System Crafters channel) spotted the error in a recent live stream whose main topic was about Denote (thanks, by the way!): https://www.youtube.com/watch?v=QcRY_rsX0yY.
  • Addressed an inconsistency in the command denote-link-find-file where it would not recognise links without a title in their format (those can be inserted by passing a prefix argument (C-u by default) to the commands that insert links, such as denote-link).
  • Attached conditionality to the denote command's SUBDIRECTORY argument, so that it does not create new file paths. This is only relevant for those who call denote from Lisp. Interactive use is the same as before.
  • Clarified that the user option denote-org-capture-specifiers can accept arbitrary text in addition to the formatting specifiers that Org's capture mechanism introduces.
  • Explained in the manual why denote-org-capture-specifiers is needed instead of writing the capture template directly the way one normally does. The gist is that because our file names are derived dynamically based on user input, we need to account for the sequence in which the value of arguments is reified by org-capture.
  • Refactored how notes are prepared internally. Thanks to Jean-Philippe Gagné Guay for the contribution in pull request 55 on the GitHub mirror: https://github.com/protesilaos/denote/pull/55.

  • Declared the denote-punctuation-excluded-extra-regexp variable which is, for the time being, targeted at experienced users. Its purpose is to extend what we consider "illegal" punctuation for the file name. Thanks to pRot0ta1p for the feedback in issue 57 over at the GitHub mirror: https://github.com/protesilaos/denote/issues/57. Example based on the input of pRot0ta1p: 

      (setq denote-punctuation-excluded-extra-regexp
        "[『』〖〗｛｝「」【】〔〕［］（）《》〈〉«»！＃￥％…＆＂＇＊，。；：、？—]*")

    The ideal is to make denote--punctuation-regexp work for all scripts, but that may be unrealistic.

  • Clarified what the manual means by "attachments" to notes. Those are for Org, if the user resorts to the relevant Org mechanisms. Denote does not do any of that.

  • Revised the parsing of a date input as used in the denote-date command or related. The idea is to turn 2020-01-15 into something like 2020-01-15 16:19 by using the current time, so that the hour and minute component is not left to 00:00 when the user does not specify it explicitly.

    This reduces the burden on the user who would otherwise need to input the time value in order to avoid the error of duplicate identifiers in the scenario where the same date is used more than once.

    The change also addresses a difference between Emacs 28 and Emacs 29 where the former does not read dates without a time component.

    Thanks to Peter Prevos for the feedback in issue 58 over at the GitHub mirror: https://github.com/protesilaos/denote/issues/58.

  • Fixed compilation warnings in Emacs 29 about the format of doc strings that need to output a literal single quote. Thanks to Kyle Meyer for the patch, which was sent on the mailing list: https://lists.sr.ht/~protesilaos/denote/patches/34117.
  • Fixed typo in the user option denote-prompts about the crm-separator. Thanks to Kyle Meyer for the patch, which was sent on the mailing list: https://lists.sr.ht/~protesilaos/denote/patches/34116.
  • Made the built-in subr-x library a runtime dependency, due to complications with the when-let* form. The problem was made manifest in a renaming operation, though it was not about renaming per se. Thanks to hpgisler for reporting the problem in issue 62 and for testing the proposed solution: https://github.com/protesilaos/denote/issues/62.
  • Streamlined the use of the seq library instead of cl-lib, as we were already using the former more heavily and there was no need for the latter. Thanks to Philip Kaludercic for pointing this out on the emacs-devel mailing list: https://lists.gnu.org/archive/html/emacs-devel/2022-07/msg00838.html.
  • Added a generic README.md file to placate the Git forges. Neither SourceHut nor GitHub/GitLab are fully compliant with the Org markup we use in README.org (we use Org because it is easy to generate the Info manual and HTML pages out of it). SourceHut will not render the file at all, while the others render it but do not parse it properly.
  • Made several other internal tweaks and refinements in the interest of robustness and/or clarity.
  • Rewrote all relevant documentation.

Non-changes

The following are not part of any changes that were made during this release cycle, though they provide potentially interesting insight into the workings of the project.

Identifiers with milliseconds
Denote's identifier format extends up to seconds. This is the product of years of experimentation and is, in my opinion, the best compromise between usability/readability and precision. If a user produces two notes within a fraction of a second, then yes they will have duplicate identifiers. In principle, there is no reason not to address this potential problem, provided we do not compromise on Denote's file-naming scheme (making the identifier less readable is a compromise). We shall see what the best course of action is. Thanks to Felipe Balbi and Jean-Philippe Gagné Guay for the discussion thus far in issue 54 on the GitHub mirror: https://github.com/protesilaos/denote/issues/54.
Denote and evil-mode
Users of evil-mode do not have to worry about Denote, as we do not define any key bindings. The manual includes sample configuration, which proposes some key bindings, but that is the user's prerogative. Thanks to Saša Janiška and Alan Schmitt for their participation on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C87czdxf1dz.fsf%40atmarama.ddns.net%3E.
Denote and Citar
Peter Prevos started developing a package that connects Denote with Citar: https://github.com/pprevos/denote-citar. The idea is to use notes as part of one's bibliography. Discussions which include sample code on how to leverage denote from Lisp:
Denote and graph of connections
Saša Janiška asked whether Denote will provide some way to visualise links between notes. The answer is negative. Denote's scope is clearly delineated and its feature set is largely complete (notwithstanding refinements to what we already provide). Peter Prevos is experimenting with some code that uses the R language. Any such facility will have to be implemented as a separate package. I remain at the disposal of anyone who needs help with Denote's internals. Thanks to the aforementioned fellows for their participation on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C878roleze1.fsf%40atmarama.ddns.net%3E.
Denote's scalability
There was a discussion whether Denote will work well with very large sets of files. The short answer is that it will work the same way Emacs and/or standard Unix tools do: good enough! If there are improvements to be made, which do not jeopardise the principles of the project, we shall implement them without hesitation. Thanks to Saša Janiška and Peter Prevos for their participation on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C87sfmtf7im.fsf%40atmarama.ddns.net%3E.
Denote's minimum requirement of Emacs 27.2

We cannot depend on Emacs 27.1 due to this message from the byte compiler: 

You should depend on (emacs "27.2") or the (org "9.3") package if you need `org-link-open-as-file'.

Depending on Org is not an option because Denote optionally works without Org, so Emacs 27.2 is what we have to opt for. If your operating system does not provide this version in package format, please petition its maintainers/providers to do so. Thanks to Alexander for asking about it on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C9ec818e6a7979efbb2f8b1f5a497665b%40purelymail.com%3E.

Finally, a mildly interesting piece of trivia: we have exceeded 600 commits since the first day of the project's Git history on 2022-06-04 (the actual history is much longer). That averages to more than 10 per day! I think things will slow down eventually.

Version 0.3.0 on 2022-07-11

  • Fixed how references are analysed to produce the backlinks' buffer. This should resolve the issue that some users faced where the backlinks would not be produced.

    The previous implementation would not yield the appropriate results if (i) the value of the user option denote-directory was a "project" per the built-in project.el and (ii) the link to the given entry was from a subdirectory. In short, the references were sometimes returned as relative file paths, whereas they should always be absolute. Thanks to Jean-Philippe Gagné Guay for the feedback in issue 42 over at the GitHub mirror: https://github.com/protesilaos/denote/pull/42.

    [ Jean-Philippe has assigned copyright to the Free Software Foundation. It is a prerequisite for contributing to core Emacs and/or any package distributed via the official GNU ELPA. ]

  • Addressed a regression in the function denote-directory (this is the function that normalises the variable of the same name) which prevented it from returning an expanded file path. This too contributed to problems with the backlinking facility. Thanks to Jean-Philippe Gagné Guay for the contribution in pull request 44 over at the GitHub mirror: https://github.com/protesilaos/denote/pull/44.

    Also thanks to user pRot0ta1p for the relevant feedback in issue 43 (also on the mirror): https://github.com/protesilaos/denote/issues/43. More thanks to Alfredo Borrás, Benjamin Kästner, and Sven Seebeck for their comments in a related thread on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3CCA73E705-1194-4324-9962-70708C4C72E5%40zoho.eu%3E. These discussions showed we had a problem, which we managed to identify.

  • Introduced the user option denote-prompts (read its doc string or the relevant entry in the manual). It governs how the standard denote command for creating new notes will behave in interactive usage. By default, denote prompts for a title and keywords. With denote-prompts, the command can also ask for a file type (per denote-file-type), subdirectory of the denote-directory, and a specific date+time. Prompts occur in the order they are specified. Furthermore, the denote-prompts can be set to values which do not include the title and keywords. This means that the resulting file names can be any of those permutations: 

    DATE.EXT
DATE--TITLE.EXT
DATE__KEYWORDS.EXT

    Recall that Denote's standard file-naming scheme is defined as follows (read the manual for the details): 

    DATE--TITLE__KEYWORDS.EXT

    For our purposes, Denote will work perfectly fine for linking and backlinking, even if file names do not include the TITLE and KEYWORDS fields. However, the user is advised to consider the implications on usability: notes without a descriptive title and/or useful keywords may be hard to filter and practically impossible to manage at scale. File names without such information should at least be added to subdirectories which themselves have a descriptive name.

    At any rate, Denote does not have strong opinions about one's workflow. The standard file name is the culmination of years of experience.

    Consider the denote-prompts the affirmative answer to the question "Can keywords be optional?" as posed by Jack Baty on the mailing list: https://lists.sr.ht/~protesilaos/denote/%3C8D392BC3-980A-4E5B-9480-D6A00BE8279F%40baty.net%3E.

    Thanks to Jean-Philippe Gagné Guay for the original contribution in commit 9b981a2. It was originally part of a pull request, but due to some internal changes I had to merge it as a patch and technically the web UI did not count the PR as "merged" (though it was in terms of substance).

  • Refactored the denote command to (i) accommodate the new user option denote-prompts via its interactive specification and (ii) be more flexible when called from Lisp. The latter scenario is for advanced users or, generally, those who can maintain some custom code in their configuration. A case in point is one of the examples we show in the manual for a programmatic way to create notes that automatically get the journal tag: 

      (defun my-denote-journal ()
    "Create an entry tagged 'journal', while prompting for a title."
    (interactive)
    (denote
     (denote--title-prompt)
     '("journal")))

    Notice that the '("journal") is a list of strings even for a single keyword. Whereas before a single one was a plain string. This is a breaking change.

    Please consult the doc string of the denote command for the technicalities.

  • Refashioned the interactive convenience functions of denote-type, denote-date, denote-subdirectory to leverage the denote-prompts user option while calling denote interactively. In practical terms, they no longer accept any arguments when called from Lisp. Users who need a programmatic approach are advised to either call denote directly, or check how these commands let bind the denote-prompts to carry out their operations. The doc string of each command explains how it works. Or evaluate this to check the manual: 

      (info "(denote) Convenience commands for note creation")

    Else visit: https://protesilaos.com/emacs/denote#h:887bdced-9686-4e80-906f-789e407f2e8f

  • Documented how the user option denote-directory can accept a local value. This is pertinent to scenaria where the user needs to maintain separate directories of notes. By "separate" we mean sets of notes that do not communicate with each other, cannot create links between them, etc. The manual delves into the technicalities. If you have the Info entry installed, evaluate: 

      (info "(denote) Maintain separate directories for notes")

    Else visit: https://protesilaos.com/emacs/denote#h:15719799-a5ff-4e9a-9f10-4ca03ef8f6c5.

    Thanks to user "Summer Emacs" for starting the discussion on the mailing list, and Benjamin Kästner for their participation: https://lists.sr.ht/~protesilaos/denote/%3Cm25yk5e856.fsf@gmail.com%3E.

  • Added an entry to the manual's Frequently Asked Questions about a failed search for backlinks. It includes sample code that users of Windows can apply, if necessary. (The error is not Denote's fault.) Thanks to Benjamin Kästner for the patch, which is below the ~15 line threshold and thus does not require copyright assignment to the Free Software Foundation: https://lists.sr.ht/~protesilaos/denote/%3Cce117b14-55cf-622e-6cd8-0af698091ae3%40gmail.com%3E.
  • Removed duplicate entries from the list of file paths that the xref library returns for the purposes of backlinking. Thanks to Jean-Philippe Gagné Guay for the contribution in pull request 44 on the GitHub mirror: https://github.com/protesilaos/denote/issues/44.
  • Applied an appropriate face to the backlinks' button to mitigate an error. Thanks to Jean-Philippe Gagné Guay for the contribution in pull request 45 on the GitHub mirror and for later testing a subsequent tweak: https://github.com/protesilaos/denote/issues/45.
  • Simplfied all the faces we define to make them work with all themes. The previous colours were consistent with the modus-themes: https://protesilaos.com/emacs/modus-themes.
  • Refined how strings are sluggified under all circumstances. Before, a nil value for the user option denote-allow-multi-word-keywords would have the adverse effect of joining all the strings in the title field of the file name. The intent always was to do that only for multi-word keywords, not the title. This change was part of a hotfix, formalised as version 0.2.1 a day after the release of 0.2.0.
  • Made the fontification rules more robust, while avoiding any false positives. This was done over a series of commits as it had implications for the file name permutations that were mentioned earlier. Thanks to Jean-Philippe Gagné Guay for the patches and/or discussion about the merits of each change and concomitant considerations:
  • Rewrote all relevant entries in the manual to reflect all the user-facing aspects of the aforementioned.

  • Discussed a use-case of rewriting old journal entries as Denote-style files. As of this writing, we do not support migration of files in bulk. It might happen at some point, though it is no mean task. Thanks to Summer Emacs and Alan Schmitt for their participation: https://lists.sr.ht/~protesilaos/denote/%3Cm27d4mbktj.fsf%40gmail.com%3E.

    An aside here as this topic was brought up: my packages are open to users of all skill levels and is why I maintain a mailing list as well as mirrors of the official git repository on SourceHut. Do not hesitate to ask a question. If, for whatever reason, those communication channels are not appropriate, you are welcome to contact me in private: https://protesilaos.com/contact.

Thanks again to Jean-Philippe Gagné Guay for the numerous contributions. Please read the commit log for the minutia, as this change log entry omitted some of the finer yet important details.

Version 0.2.0 on 2022-07-04

  • Version 0.1.0 (from 2022-06-27) was never built as a package. The reason is that the GNU ELPA machinery reads the Version: header of the main file, not the git tag. As the original commit in denote.el included Version: 0.1.0, GNU ELPA rightly tries to build the package using that reference. But because at that time I had not yet updated the Copyright header to name the Free Software Foundation, the package could not be prepared. As such, please consider this release to be the "first formal stable version". My apologies for the delay, contrary to what was promised in the last change log entry.
  • Originally, Denote was designed to only work with notes in a flat directory. With code contributions from Jean-Philippe Gagné Guay, support for subdirectories of the user option denote-directory is now available. This covers the case of creating links between notes, following them, and viewing the backlinks' buffer of the current entry.
  • The new denote-subdirectory command lets the user select a directory to place the new note in. Available candidates are the value of the denote-directory as well as all of its subdirectories, minus .git. In future versions, we will consider how to provide a blocklist or a regexp filter for the user to specify which subdirectories should be omitted from minibuffer completion. Please consider providing your feedback on the technicalities.
  • From 2022-06-24 to 2022-07-03, Denote provided support for links between Org notes that leveraged the id: hyperlink type. Discussions on the mailing list and the GitHub mirror revealed the longer-term problems in our implementation. In the Annex below, I provide my detailed opinion on the matter. The gist is that Denote does not—and will not—create id: links between its notes, but shall use the denote: hyperlink type instead (which works like the standard file: type). As the Annex explains, Denote is not org-roam lite and we try not to engender such false expectations.

  • The user option denote-date-format controls how the date and time is recorded in the file's contents (what we call "front matter"). When nil (the default value), we use a file-type-specific format (also check the user option denote-file-type):

    • For Org, an inactive timestamp is used, such as [2022-06-30 Wed 15:31].
    • For Markdown, the RFC3339 standard is applied: 2022-06-30T15:48:00+03:00.
    • For plain text, the format is that of ISO 8601: 2022-06-30.

    If the value is a string, ignore the above and use it instead. The string must include format specifiers for the date. These are described in the doc string of format-time-string.

    The denote-date-format supersedes the now obsolete denote-front-matter-date-format.

    Thanks to Peter Prevos and Kaushal Modi for their feedback in issue 27 on the GitHub mirror: https://github.com/protesilaos/denote/issues/27.

  • All the faces we define are now declared in the denote-faces.el file. The fontification rules are shared by denote-dired-mode and the backlinks' buffer (invoked by denote-link-backlinks and controlled by the user option denote-link-fontify-backlinks). The current list of faces:
    • denote-faces-date
    • denote-faces-delimiter
    • denote-faces-extension
    • denote-faces-keywords
    • denote-faces-subdirectory
    • denote-faces-time
    • denote-faces-title
  • Named the mailing list address as the Maintainer: of Denote. Together with the other package headers, they help the user find our primary sources and/or communication channels. This change conforms with work being done upstream in package.el by Philip Kaludercic. I was informed about it here: https://lists.sr.ht/~protesilaos/general-issues/%3C875ykl84yi.fsf%40posteo.net%3E.
  • Fixed how keywords are inferred and combined. The previous code did not work properly when the user option denote-infer-keywords was nil. It would return a list of symbols, with the parentheses, whereas the file name needs a string where each keyword is delimited by an underscore.
  • Simplified how information in the front matter is retrieved. It fixes cases where, for example, a special character at the end of the title was ignored. Thanks to Jean-Philippe Gagné Guay for the patch over at the GitHub mirror: https://github.com/protesilaos/denote/pull/21.
  • Rewrote parts of the manual in the interest of clarity.

Annex about discontinuing support for org-id

My thanks for their participation in the discussions go to Jean-Philippe Gagné Guay, Kaushal Modi, and Shreyas Ragavan. 

commit f35ef05cb451f265213c3aafc1e62c425b1ff043
Author: Protesilaos Stavrou <info@protesilaos.com>
Date:   Sun Jul 3 17:34:38 2022 +0300

    REMOVE support for 'id:' hyperlink types

    The original idea was to support the 'org-id' library on the premise
    that it makes Denote a good Emacs citizen.  However, discussions on the
    mailing list[0] and the GitHub mirror[1] have made it clear to me that
    'org-id' is not consistent with Denote's emphasis on simplicity.

    To support the way 'org-id' works, we will eventually have to develop
    some caching mechanism, just how the org-roam package does it.  This is
    because the variable 'org-id-extra-files' needs to be kept up-to-date
    whenever an operation on a file is performed.  At scale, this sort of
    monitoring requires specialised software.  Such a mechanism is outside
    the scope of Denote---if you need a db, use org-roam which is already
    great.

    [0] <https://lists.sr.ht/~protesilaos/denote/%3C8735fk4y1w.fsf%40hallac.net%3E#%3C877d4un73c.fsf@protesilaos.com%3E>

    [1] <https://github.com/protesilaos/denote/issues/29>

    Quote of what I wrote on the GitHub mirror issue 29:

            [ggjp] This is what I was implying.  That we are, in fact,
            providing an option that is not viable long-term, but keeping
            the option for expert users who will be able to handle this.
            And we should warn about this clearly in the doc of that option.

        [protesilaos] What you write here @ggjp and what @shrysr explained
        tells me that those expert users will need to be real experts.  To
        put it concretely, I am an experienced Emacs user with no
        programming background, who has written several Emacs
        packages (including the modus-themes which are built into Emacs),
        but I have zero knowledge of using a db or of handling things with
        python and the like.  So if I opt in to 'denote-link-use-org-id' I
        will eventually run into problems that my non-existent skills will
        prevent me from solving.  At that point, I will just use org-roam
        which already handles this use-case in a competent way (and has a
        massive community to rely on in case I need further support).

        If each package needs to write its own optimisations and maintain
        its own cache, to me this shows that 'org-id' is not good enough for
        the time being: more work needs to be done in org.git to provide a
        universal solution.

        I wanted to support 'org-id' by default on the premise that Denote
        must be a good Emacs citizen which interoperates with the rest of
        the wider ecosystem.  But if 'org-id' leaves something to be
        desired, then that goal is not worth pursuing: we add complexity to
        our code, offer an option that we cannot genuinely/adequately
        support, and make usage of it contingent on reading the docs and
        having a high level of expertise.

        I think being a good Emacs citizen is a laudable principle.  In this
        case, the right thing to do is to recommend the use of org-roam
        instead of trying to accommodate 'org-id'.  As such, I have now
        changed my mind and think we should remove what we previously added.

        For some context here: the reason I never used org-roam is
        because (i) it is Org-specific whereas I write notes in different
        file types and (ii) I did not want to ever rely on a db or
        equivalent dependency.

        <https://github.com/protesilaos/denote/issues/29#issuecomment-1173036924>

 README.org         | 226 ++++++++---------------------------------------------
 denote-link.el     |  99 ++++++-----------------
 denote-retrieve.el |   2 +-
 denote.el          |  14 +---
 4 files changed, 63 insertions(+), 278 deletions(-)

Followed up by my explanation: 

> can we not have denote style links to be default for (de)notes - and
> explicitly supported, while if they need to, users can still link
> denote org files via org-id to any other notes/files (and vice versa)
> -- in which case performance + testing for org-id driven linking is
> not within Denote's purview at all?

The formal support for `id:` links was added shortly before the release
of version `0.1.0`.  In the days prior, we supported what you describe
via the manual.  The user could change the `denote-org-front-matter`
variable to include a `PROPERTIES` drawer.  This possibility still
exists, though yesterday I removed the relevant entry from the manual.
This way only the real do-it-yourself experts will go down that path.

My concern here is with managing expectations.  If our Org notes are
superficially the same as org-roam's, an unsuspecting user may think
that Denote is an org-roam lite.  We will thus get issues/requests, such
as those already mentioned in this GitHub repo, about migrating from
org-roam to Denote.  While there are similarities, Denote is not a
minimalist org-roam and I would not like to encourage the idea of
treating the two as interchangeable.

Doing things half-way-through is a way to create false expectations.  A
package on GNU ELPA must be usable by users of all skill levels.  If the
functionality we provide is incomplete and needs to be covered by
user-level tweaks, we are excluding a portion of the user base while
still assuming the maintenance burden.  If someone trusts Denote to,
say, write a 1000 notes, we do not want to surprise them after the fact.
Imagine if the reported issues that triggered this change happened 6
months into one's daily usage of Denote: it wouldn't be nice.

Setting the right expectations is a matter of responsibility: we let the
user make a more informed choice and show respect for their time.  It
also makes it easier for me to keep Denote's scope in check by not
supporting every little extra that Org implements.  The premier Org
extension is org-roam: we do not need another one (or, if we do, I am
not the one to implement it).

* * *

Some comments on the `denote:` hyperlink type for Org as they may be
relevant in this context:

* It is meant to work like the standard `file:` type.  This means that
  it links to a file, while it can also have additional search
  parameters, as explained in the Org manual.  Evaluate:

      (info "(org) Search Options")

* It does not read the front matter, but only the file name.  You can
  create a note as usual, delete all its contents, save it, and try to
  link to it from another note.  It works.

* Exporting now works like the `file:` type for HTML, LaTeX, Texinfo,
  and Markdown.  Technically, it also supports the ASCII backend but the
  format of the output could be tweaked further.

There may be refinements to be made, which is okay as that is part of a
maintainer's duties.

Version 0.1.0 on 2022-06-27

The present entry is intended for early adopters of Denote who may have not caught up with the latest developments. Prospective users are advised to read the manual: https://protesilaos.com/emacs/denote. For a video demonstration: https://protesilaos.com/codelog/2022-06-18-denote-demo/.

  • The denote package on GNU ELPA will be available a few hours after this release. GNU ELPA provides the latest stable release. To use a development snapshot, read: https://protesilaos.com/codelog/2022-05-13-emacs-elpa-devel/.
  • Remember that any significant contribution (above ~15 lines) requires copyright assignment to the Free Software Foundation. A form with instructions is included in the manual's "Contributing" section: https://protesilaos.com/emacs/denote#h:1ebe4865-c001-4747-a6f2-0fe45aad71cd.

  • The front matter of notes in Org has changed to be compliant with the standard org-id infrastructure. A PROPERTIES drawer is added to the top of the file, which includes an ID property with the value of the Denote identifier. Sample: 

      :PROPERTIES:
  :ID:          20220610T202537
  :END:
  #+title:      Sample Org front matter
  #+date:       2022-06-10
  #+filetags:   denote  testing

  • The front matter of Markdown (YAML or TOML) and plain text files remains constant. For completeness, this is how they look: 

       ---
   title:      "Sample with Markdown and YAML"
   date:       2022-06-10
   tags:       denote  testing
   identifier: "20220610T202021"
   ---

      +++
  title      = "Sample with Markdown and TOML"
  date       = 2022-06-10
  tags       = ["denote", "testing"]
  identifier = "20220610T201510"
  +++

      title:      Sample plain text
  date:       2022-06-10
  tags:       denote  testing
  identifier: 20220610T202232
  ---------------------------
  • The integration with org-id extends to how linking works. By default, Denote uses its own custom hyperlink type which starts with the denote: prefix. In Org, it works like the file: type. When the user option denote-link-use-org-id is non-nil, links from Org notes to other Org notes will use the standard id: type instead. As this is an Org-specific feature, Denote takes care to use the major-mode-agnostic denote: type when the link targets a non-Org note.

  • In Org files the links created by Denote are buttonized automatically. For Markdown and plain text, we use our own methods. When a link is inserted it is buttonized outright. To buttonize links in existing notes while visiting them in a buffer, add/evaluate this (it excludes Org on its own): 

      (add-hook 'find-file-hook #'denote-link-buttonize-buffer)
  • The generation of the backlinks' buffer now uses the built-in xref library instead of relying on a hardcoded call to the find executable. This means that the denote-link-backlinks command will, in principle, work properly with all Emacs builds.
  • Users of Emacs 28 or higher can configure xref-search-program to change from the default grep to ripgrep, ugrep, or a user-defined alternative.
  • This is the first stable release of Denote. It covers close to 400 commits starting from 2022-06-04. Denote is the successor to a toy package of mine, USLS, whose first public version was made available in early November 2020: https://gitlab.com/protesilaos/usls.
  • Thanks to everyone involved in the development of Denote. Code contributions, bug reports, discussion of ideas, are all valuable. From A-Z the names mentioned in the manual's "Acknowledgements" section: Colin McLear, Damien Cassou, Frank Ehmsen, Jack Baty, Kaushal Modi, Peter Povinec, Sven Seebeck, Ypot.
  • Sources of Denote: