@david@collantes.us Such a funny picture – we’ve been to Florida once some ~30 years ago and it looked almost exactly like that. 😅

@david@collantes.us Yeah, but it happened so fast with him. 😅 I remember watching some of his talks 1-3 years ago, looked completely different, I think. 😅

Luckily I can still recognize the voice, so I know it’s him, lol.

@lyse@lyse.isobeef.org The hash/thread-id would be shorter, but you’d lose two other benefits of (replyto:…):

1. You need a special client again to compute hashes.
   
2. The original feed URL is no longer visible, thus you might need to ask a Yarn pod occasionally for missing twts (I do that surprisingly often, now that I’ve implemented it) – but now you’ve lost the guarantee that Yarn gives you the correct information, because you can no longer verify it.

@lyse@lyse.isobeef.org Right, feed rotation gets ugly. We’d have (replyto:example.com/tw.txt,$timestamp) but maybe that feed doesn’t actually contain that stamp, so you have to got further back … but you should NOT reference an archived feed in your (replyto:…) thingy, it should still be the “main feed URL” (because the contents of archived feeds aren’t stable, see @prologic@twtxt.net’s feeds for example). That’s not too great.

Man, I’m completely torn on this. I’d almost prefer not to decide anything. 😂

… then, of course, I wouldn’t need to ask a Yarn pod for a certain twt if we used (replyto:…) instead of (#123467), because the original source of the twt is no longer obscured by a hash value and I can just pull the original feed. Asking a Yarn pod is only interesting at the moment because I have no idea where to get (#123467) from.

Only when the original feed has gone offline will querying a Yarn pod become relevant again.

I have to admit here that some of the goals/philosophy of Yarn simply don’t apply to my use cases. 😅 I don’t run a daemon that speaks a gossipping protocol with neighboring pods or stuff like that. I think I don’t have a hard time accepting that feeds might go offline in two months, so be it. Digging up ancient twts from some sort of globally distributed file system isn’t one of my goals. It’s a completely different thing for me. Hmmm. 🤔

I’m bad with faces, I know that. But I’m having a really hard time recognizing Linus in this video:

https://www.youtube.com/watch?v=4WCTGycBceg

Basically a different person to me. Is it just me or has he really changed that much? 😳

@david@collantes.us Glad you like it. 😅

@david@collantes.us Aye, I’ve pushed some commits. (And this is really going to be the last non-trivial change. 😂)

@david@collantes.us Like that, right? https://movq.de/v/80f888d381/s.png

Okay, the recently implemented --fetch-context, which asks a Yarn pod for a twt, wouldn’t break, but jenny would not be able anymore to verify that it actually got the correct twt. That’s a concrete example where we would lose functionality.

@david@collantes.us Yeah, I was annoyed by this myself lately. twts have become so long nowadays, it really gets in the way.

@prologic@twtxt.net Can you come up with actual scenarios where it would break? Or is it more of a gut feeling?

The thing that keeps bugging me is this:

If we were to switch to location-based addressing and (replyto:…), the edit problem would resolve itself. Implementations could use that exact string (e.g., https://example.com/tw.txt,2024-09-18T12:45Z) as the internal identifier of a twt and that is pretty much the only change that you have to make. And then you could throw away all code and tests currently required for calculating hashes. (In jenny, I would also be able to and actually have to remove that code that skips over twts with a timestamp older than $last_fetch. This only got added as a workaround “to avoid broken threads all the time”.) The net result would be less code.

Implementing this whole (edit:#hash) thing means more code. (For jenny, specifically, a lot more code, if I want to allow users to create such twts.)

Do you see why I’m so reluctant to jump on this bandwagon? 😅

I haven’t come up yet with good, concrete examples where (replyto:…) would break. As soon as that happens, I’ll change my mind. 🤔

For implementations, it would be nice if “update twts” always came after the twt they are referring to. So I thought about using this opportunity to mandate append-style feeds. But that’s just me being lazy. Implementations will have to be able to cope with any order, because feeds cannot/should not be trusted. 🫤

Trying to sum up the current proposal (keeping hashes):

1. Extend the hash length to avoid collisions.
   
2. Introduce the concept of, what shall we call it, “update twts”.
   
   • A twt starting with (edit:#3f36byq) tells clients to update the twt #3f36byq with the content of this particular twt.
     
   • A twt starting with (delete:#3f36byq) advises clients to delete #3f36byq from their storage.
     

Right?

@prologic@twtxt.net

you’d never have been able to find let alone pull up that ~3yr old Twt of me (my very first), hell I’d even though I lost my first feed file or it became corrupted or something

I get what you mean, but to be fair, it’s much less mysterious than that. 😅 The twt in question exists in your archived feed. It’s not like I pulled it out of some cache of an unrelated Yarn pod.

But, yes, I could have done that and I could have verified that it actually is the twt I was looking for. So that’s clearly an advantage of the current system.

(Or maybe I’m talking nonsense. That’s known to happen. I’ll go to bed. 😂)

@quark@ferengi.one Printing a version? I’ll think about it. 🤔

It would be easy to do for releases, but it’s a little hard to do for all the commits in between – jenny has no build process, so there’s no easy way to incorporate the output of git describe, for example.

I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.

The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.

I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.

If twtxt/Yarn was to grow bigger, then this would become a concern again. But even Mastodon allows editing, so how much of a problem can it really be? 😅

I do have to “admit”, though, that hashes feel better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.

Hm.

I suspect that the (replyto:…) proposal would work just as well in practice.

@falsifian@www.falsifian.org @prologic@twtxt.net @lyse@lyse.isobeef.org

• editing, if you don’t care about message integrity
  

So that’s the big question, because that’s the only real difference between hashes and the (replyto:…) proposal.

Do we care about message integrity?

With (replyto:…), someone could write a twt, then I reply to it, like “you’re absolutely right!”, and then that person could change their twt to something malicious like “the earth is flat!” And then it would look like I’m a nutcase agreeing with that person. 😅

Hashes (in their current form) prevent that. The thread is broken and my reply clearly refers to something else. That’s good, right?

But now take into account that we want to allow editing anyway. Is there even a point to using hashes anymore? Isn’t message integrity ignored anyway now, at least in practice?

There’s no difference (in practice) between someone writing

2024-09-18T12:34Z    Brds are great!

and then editing it to either

2024-09-18T12:34Z    (original:#12379) Birds are great!  (Whoops, fixed a typo.)

or

2024-09-18T12:34Z    (original:#12379) The earth is flat!

The actual original message is (potentially) gone. The only thing that we can be sure of now is that the twt was edited in some way. Essentially, the actual twt message is no longer part of the hash, is it? What does #12379 refer to? The edited message or the original one? We want it to refer to the edited one, because we don’t want to break threads, so … what’s the point of using a hash?

Regarding jenny development: There have been enough changes in the last few weeks, imo. I want to let things settle for a while (potential bugfixes aside) and then I’m going to cut a new release.

And I guess the release after that is going to include all the threading/hashing stuff – if we can decide on one of the proposals. 😂

@aelaraji@aelaraji.com Looks like your shell didn’t turn the \n into actual newlines:

$ echo -n "https://twtxt.net/user/prologic/twtxt.txt\n2020-07-18T12:39:52Z\nHello World! 😊" | openssl dgst -blake2s256 -binary | base32 | tr -d '=' | tr 'A-Z' 'a-z' | tail -c 7
zq4fgq
$ printf "https://twtxt.net/user/prologic/twtxt.txt\\n2020-07-18T12:39:52Z\\nHello World! 😊" | openssl dgst -blake2s256 -binary | base32 | tr -d '=' | tr 'A-Z' 'a-z' | tail -c 7
p44j3q

@quark@ferengi.one They’re all RFC3339, unless I’m mistaken: https://ijmacd.github.io/rfc3339-iso8601/ So they’re all correct.

@prologic@twtxt.net So the feed would contain two twts, right?

2024-09-18T23:08:00+10:00	Hllo World
2024-09-18T23:10:43+10:00	(edit:#229d24612a2) Hello World

@prologic@twtxt.net I don’t get paid for “standing by” and “waiting for a call”, that’s right. But I’m fine with that, because I don’t have to be available, either. 😅 If someone were to call me (or send me a text message), I wouldn’t be obliged to help them out. If I have the time and energy, I will do it, though. And that extra time will be paid.

It works for us because there are enough people around and there’s a good chance that someone will be able to help.

Really, I am glad that we have this model. The alternative would be actual on-call duty, like, this week you’re the poor bastard who is legally required to fix shit. That’s just horrible, I don’t want that. 😅

What I was referring to in the OP: Sometimes I check the workphone simply out of curiosity. 😂

@prologic@twtxt.net text/plain without an explicit charset is still just US-ASCII:

The default character set, which must be assumed in the absence of a charset parameter, is US-ASCII.

https://www.rfc-editor.org/rfc/rfc2046.html#section-4.1.2
https://www.rfc-editor.org/rfc/rfc6657#section-4

@lyse@lyse.isobeef.org Ouch. 🥴 Well, jenny always decodes as UTF-8 (because the spec says so) and this hasn’t caused any issues – yet.

@prologic@twtxt.net It’s not “true” oncall duty (I don’t do that anymore, it fucks up your life and health), but more of an unwritten rule that sysadmins should have a smartphone. If there is an emergency and I happen to have time to help, I will. But nobody can force me to. So far, it works for us.

And, well, I also need that thing for 2FA. 😅

@prologic@twtxt.net I’m pretty much required to have one. 🫤

@prologic@twtxt.net No, wait, I removed something else. But I think it was never supported in the first place. 😅 Probably because nobody used it (which is probably because you didn’t implement a UI for them 😅).

When will I learn to not look at my work’s smartphone in my free time? 😂 Opened the corporate chat, instant regret.

@prologic@twtxt.net Weeeeeeeeeeeeelllll … I have removed support for that in jenny about a month ago. 🥴 Nobody ever used them, only hashes.

@lyse@lyse.isobeef.org One of them is lying. >:-)

Mangled in what way? How does it look like? 🤔

@cuaxolotl@sunshinegardens.org

i hope we don’t go chasing mass-appeal

I don’t think this is going to happen any time soon.

What makes twtxt unique is its radical technical simplicity. And that means you have to be a tech-savvy person to appreciate twtxt and that means mass-appeal is pretty much out of the question to begin with. 😅

Yarn adds a lot of user-friendliness and, in a way, does try to appeal to the masses. And it almost worked. When Twitter died, we saw a substantial influx of new users, didn’t we? Only problem is, Mastodon had a massive headstart. The Fediverse was already huge, so what did Yarn/twtxt have to offer? It is way simpler and way easier to self-host, but who cares about that? Right, the tech-savvy people, not the masses. Many of the tech-savvy people were already using Mastodon, though, and, frankly, “simplicity” is not something that a lot of folks even care about. Thus Yarn/twtxt never took off.

A nice afternoon. Mild weather (~23°C), sitting on the balcony, working a bit on jenny, and spamming twt’s. 😅

@quark@ferengi.one At the moment, the twt in question exists in the sixth archive:

$ jenny -D https://twtxt.net/user/prologic/twtxt.txt/6 | head
[o6dsrga] [2020-07-18 12:39:52+00:00] [Hello World! 😊]


Does that work for you? 🤔

@prologic@twtxt.net Yeah, that thing with (#hash;#originalHash) would also work.

Maybe I’m being a bit too purist/minimalistic here. As I said before (in one of the 1372739 posts on this topic – or maybe I didn’t even send that twt, I don’t remember 😅), I never really liked hashes to begin with. They aren’t super hard to implement but they are kind of against the beauty of the original twtxt – because you need special client support for them. It’s not something that you could write manually in your twtxt.txt file. With @sorenpeter@darch.dk’s proposal, though, that would be possible.

I don’t know … maybe it’s just me. 🥴

I’m also being a bit selfish, to be honest: Implementing (#hash;#originalHash) in jenny for editing your own feed would not be a no-brainer. (Editing is already kind of unsupported, actually.) It wouldn’t be a problem to implement it for fetching other people’s feeds, though.

@quark@ferengi.one Hmm. I cannot reproduce this. 🫤 I just removed all files in ~/.cache/jenny and ~/Mail/twt/cur, and a subsequent jenny -f properly fetches everything.

Do you see all the “Fetching archived feed …” messages?

@prologic@twtxt.net

Your propose scheme while simple doesn’t do this.

It doesn’t do that because it’s not taking the content of a twt into account (only its timestamp). Okay. But the mere fact that we’re talking about “how to solve the edit problem” stems from using content addressing – so maybe content addressing isn’t the best thing to use here? 🤔

@prologic@twtxt.net I’m very torn on this.

It’s a cool idea and it’s cool technology. It would (probably) even be fun to implement.

But do we need it? Or rather, does twtxt need it? What problem are you trying to solve – are people migrating their feeds to new URLs all the time? 🤔 That’s rather rare in my experience. The URL as the primary identifier of a feed works fine for me.

Maybe I just don’t understand the problem well enough yet? 🤔

@quark@ferengi.one

jenny does fetch archived feeds during the normal jenny -f operation […]

… but you need to use the current Git version which includes this commit:

https://www.uninformativ.de/git/jenny/commit/6e8ce5afdabd5eac22eae4275407b3bd2a167daf.html

There was a bug that broke on @prologic@twtxt.net’s feed. 🥴

@quark@ferengi.one

Since jenny can’t fetch archived twtxts

I wiped my entire maildir and re-fetched everything. I did that recently because @aelaraji@aelaraji.com asked me to 😅, but I guess I also did this back in 2023.

What did you do to make yours work?

jenny does fetch archived feeds during the normal jenny -f operation. Only when using the recently implemented --fetch-context, archived feeds are not fetched (yet). That was an oversight and I intend to fix that.

@quark@ferengi.one Ahh, I see:

Message-Id: <o6dsrga>

That’s an older format that was used before jenny version v23.04. It should look like this nowadays:

Message-Id: <o6dsrga@twtxt>

Changelog entry from back then:

v23.04  2023-04-19
  [Changed]
  - The format of the "Message-Id" and "In-Reply-To" headers has
    changed. They now need an "@twtxt" suffix to be more compliant with
    RFC(2)822. This fixes issues when using aerc
    (https://aerc-mail.org/) as a frontend instead of mutt.

    If you want to retain compatibility with existing files in your
    maildir, you must manually add this suffix to these headers. (Or go
    ahead and re-sync everything.)

I guess I could have added backwards compatibility to the code. Maybe I’ll fix that later. 🤔

@aelaraji@aelaraji.com 🥳

@bender@twtxt.net Ah, haha, --fetch-context doesn’t go back into archived feeds … 🤦

@prologic@twtxt.net

@quark@ferengi.one This is what I get. 🤔

@lyse@lyse.isobeef.org Argh. Yeah. Well. 🤦

The bug in jenny that @aelaraji@aelaraji.com found:

Jenny has to look for the metadata fields, it must find the # prev = ... line. To do so, I naively wrote something along these lines:

for line in content.splitlines():
    if line.startswith('# prev = '):
        ...

Problem is, we use \u2028 a lot in twtxt feeds and Python interprets those as line separators as well. That’s not what we want here. Jenny must only split at a \n.

Now @prologic@twtxt.net had a quote/copy of some of his metadata fields in a twt. Like so:

# prev = foo bar

Perfectly legitimate, but now jenny found the # prev = twice (once in the actual header, once in a twt), didn’t know what to do, and thus did not fetch the archived feeds. 🤦

Should be fixed in this commit: https://www.uninformativ.de/git/jenny/commit/6e8ce5afdabd5eac22eae4275407b3bd2a167daf.html

This is a bug in jenny. 🤦

@prologic@twtxt.net Oh so that’s how it works? The front page only shows the latest twt of each feed? 🤔

No, something is fishy. It didn’t fetch @prologic@twtxt.net’s archived feeds and now only 969 of his twts are in my maildir. 🤔