feed digester

the purpose of this program is to time-and-location shift all the online stuff you peruse, according to your predilictions. it consumes a collection of sources eg. rss, activitypub, gemfeeds, etc and redirects the posts found therein to one or more destinations eg. an epub, wallabag, email, an xmpp message, etc.

in order to use it, one defines some sources, and some destinations, and a set of rules which route posts from the defined sources to the defined destinations.

config

the feed digester config is written in the edn data format[a]. and is stored in <config_path>/config.edn. an example config:

{:sources [
  {:id :fedi
   :type :mastodon
   :domain "example.net"
   :username "sally"
   :path ""}
  {:id :fedi-books
   :type :bookwyrm
   :domain "reading.example.net"
   :username "lucille"}
  {:id :rss-text
   :type :rss
   :path "rss-text.opml"}]

 :destinations [
  {:id :net-digest
   :type :epub}
  {:id :photography-feed
   :type :stash
   :url "https://stash.example.net/stash"
   :tags ["photography"]}
  {:id :catch-all
   :type :raw
   :path ""}]

 :rules [
  {:when [:and [:= :source :fedi]
               [:= :tags ["photography" "film"]]
               [:= :attachment-type :image]]
   :destination :photography-feed
   :exclusive true}
  {:when [:= :source [:rss-text :fedi :fedi-books]]
   :destination :net-digest
   :exclusive true}
  {:when true
   :destination :catch-all}]}

[a] edn data format

config.edn contains a hashmap with three keys.

:sources <vector[source]>
:destinations <vector[destination]>
:rules <vector[rule]>

:sources

:destinations

:rules

the :rules key references a vector of hashmaps. a hashmap requires two keys.

:when <condition>
:destination <destination-id>

when a evaluates to true, the post being processed will be sent to the referenced by .

optionally, an :exclusive key can be added. if :exclusive is true and evaluates to true, processing of the current post will halt. by default, processing of subsequent rules will continue.

:exclusive <boolean>

:when

the value of the :when key is a , and is expressed in a small domain specific language inspired by honeysql[b]. it comprises one or more vectors, in nested fashion. the first element of each vector is an operator. the result of the evaluated expression is boolean. ie

[<op> <arguments...>]

[<op> [<op> <arguments...>]
      [<op> <arguments...>]]

[b] honeysql

only three operators are supported. :and, :or and :=. the first two are boolean operators. they behave as you would expect, if you're familiar with that sort of thing. := is overloaded to some degree, in ways that don't strictly reflect equality. all operators evaluate to a boolean value. for example:

[:and [:= :source :fedi]
      [:= :tags "photography"]]

this expression evaluates to true if the source of the post being processed is :fedi and the post contains the #photograpy hashtag.

[:and <condition...>]

the :and operator must be provided one or more arguments. the arguments must be a condition. note, a condition is either a vector with a valid operator and arguments, or a boolean value.

[:and false
      [:= :source :foo]]

expressions such as this can be useful for disabling a rule without deleting it.

[:or <condition...>]

the :or operator takes the same arguments, but returns true if any arguments evaluate to true, and false, only if all arguments evaluate to false.

[:or [:= :source :fedi]
     [:= :tags "photography"]]

this will evaluate to true if the post being processed is from the source :fedi, or if it contains a photography tag.

[:= <post-key> <string|keyword|vector[string|keyword]>]

the :eq operatore requires two arguments. the first a post key, and the second a value. valid post keys are:

:source <source-id>
:tags <boolean|tag|vector[tag]>
:attachment <boolean>
:attachment-type <attachment-type>

valid attachment types are:

:image
:video

it has a few different behaviours, depending on the value argument provided.

[:= :tags true]

if the value is a boolean it tests the presence of post key. the above example evaluates to true if the post has 1 or more tags, and false otherwise.

[:= :tags "photography"]

if the value is a string it tests the presence of that value. the above example evaluates to true if the post has the photography tag. false otherwise.

[:= :tags ["photography" "film"]]

if the value is a vector it tests the presence of any of the provided values. the above example evaluates to true if the post has either the photography tag or the film tag. false otherwise.