I’ve been thinking about an RSS Reader feature for a long time now, and I really wish it existed in Google Reader. I’d love the ability to say,

Show me the top n stories out of those that remain unread.

So if I have over a thousand to read, and I just want to hit the high points, Google should be able to tell me. Here’s how.

1. They know the destination URLs for the items I could click on.
2. They know which of them has been clicked more often.
3. They know which have been shared more often.

Using that information they could rank them, and then allow people to filter based on top 10, 25, 50, 100, etc., out of those that remain unread.

This would make it far easier to play catch-up on a neglected, high-volume feed without missing the big stories. ::

This is an exciting protocol. I want to use it to get my various services’ content into Friendfeed, and to Google Feedburner, and to my website, very quickly.

I need to figure out how to implement this using [pubsubhubbub](http://pubsubhubbub.googlecode.com/ “pubsubhubbub -

Google Code”).

I mentioned recently that I had been working on my RSS feeds; what I failed to point out was that a number of the addresses changed. The base URL for all the feeds is now feeds.danielmiessler.com, and I’ve changed a few of the feed names as well to provide additional permanence.

Please delete the old ones you have and re-subscribe to these. Sorry for the hassle; this is the last part of my domain move and it should be the last time I have to do it.

Main Feeds

  | blog

  | study

  | writing

Additional Feeds

   | information security

   | technology

   | atheism

   | politics

   | discovered content

::

One of the things I’ve always found lacking with the various types of RSS feeds available from social services is the lack of differentiation between created and discovered content.

So I don’t just want to differentiate between blogs, microblogs, comments, and likes–I want to group these into categories that then have their own feeds.

Why does this matter?

It matters because the website is slowly morphing into what it will end up being, which is an interface to one’s lifestream. A lifestream is everything a person does–captured digitally–and then presented for others to consume.

In the future you’re just going to subscribe to people–not to services they use, or any of those other types of specifics–just people. And within your interface to that person you’ll be able to select what you want to see of theirs, and how.

Created vs. Found

One of the biggest differentiators in this breakdown is the difference between what that person created themselves, vs. what they found and liked. And these different types of content should be separated (or be ABLE to separated) within all interfaces to someone’s content.

I’ve been providing my own initial push in this direction via my “shared content” feed for quite some time now. It’s just an RSS feed of my “Google Shared Items” from Google Reader, rendered by Javascript. I’ve received a wealth of positive feedback on it, with many people saying they’re using it like their digg or reddit.

And if you think about that, it makes complete sense.

If I subscribe to your blog (i.e. the content YOU create), then I’m likely to also enjoy what you enjoy. Or, at least there will be some solid overlap. As such, I want to see, in very explicit terms, the things you’re finding and enjoying. This means EVERYTHING you like, from ALL your services:

• stories you vote up on reddit/digg
• albums you like on last.fm / pandora
• stuff you like from stumbleupon

etc…

Basically, everything you would want your friends to check out that you come across. In the future, you’ll be able to aggregate many of these “discovered” feeds together, for your favorite people, and you’ll assign weights to the peoples’ opinions which will prioritize the stuff you’re most likely to like based on how many of your friends recommended it, and what weight you give to each friend.

Long sentence.

Progress

Anyway, that “disovered content” feed on my site is what I’ve just enhanced. Rather than just having my Google Shared Items (which is cool, you understand) I am now posting from a Yahoo! Pipe that pulls from the RSS feed of a Friendfeed search.

The search omits stuff like twitter, comments, and other crap that isn’t “something cool I found”. But it includes every other service in my Friendfeed profile, including Pandora, Digg, Reddit, Last.fm, Stumbleupon, etc. Here’s the Friendfeed Search syntax for ommiting things:

[bash]-service:twitter -service:disqus -service:backtrack[/bash]

So basically, if I think it’s cool, it’s going into that feed–and that feed is being pulled and displayed by your browser on every load of my site. That’s one of the primary functions that every website should have: showing what you find interesting and worthwhile.

This is so important that it should be part of WordPress. It should also become a main feed of Friendfeed. And every other service that presents an interface to “you”.

That’s my opinion, anyway, and I think others will agree once they’re shown the way. ::

[ Oh, and here's the url to my discovered content feed (abstracted behind Google Feedburner) | http://feeds.dmiessler.com/lifefeed_discovered ]

I’m no longer calling feeds “RSS feeds”. From now on I’m calling them “syndication” feeds, or just “feeds”. There have been two standards for doing this type of XML-based website syndication for a long time now — Atom and RSS. RSS is older and has more penetration, so its name has been confused with its function. Just like Xerox, Kleenex and Google.

Well, I don’t use Xerox to mean copy, I don’t use Kleenex to mean tissue, and the only reason I use “google” to mean search is because it’s vastly superior to the alternatives. If that changes I’ll drop it as well.

And that’s the case with RSS. It’s not simply not superior to Atom — it’s inferior. Here are a few of the highlights:

1. Atom offers a number of technical advantages that hinge on a more precise and scalable definition. Think XHTML vs. HTML4. Examples include (from the Atom Wiki):
   • RSS 2.0 requires feed-level title, link, and description. RSS 2.0 does not require that any of the fields of individual items in a feed be present. Atom 1.0 requires that both feeds and entries include a title (which may be empty), a unique identifier, and a last-updated timestamp.
   • RSS 2.0 may contain either plain text or escaped HTML, with no way to indicate which of the two is provided, and the RSS 2.0 content model does not permit actual well-formed XML markup, which reduces the re-usability of content. Atom has a carefully-designed payload container, where content must be explicitly labeled.
   • RSS autodiscovery has been implemented several times in different ways, often relies on unregistered (thus invalid) application/rss+xml MIME type, and has never been standardized. Atom [WWW]standardizes autodiscovery. The application/atom+xml MIME Type is registered with IANA. Additionally, Atom feeds contain a “self” pointer, so a newsreader can auto-subscribe given only the contents of the feed, based on Web-standard dispatching techniques.
   • RSS 2.0 is not in an XML namespace but may contain elements from other XML namespaces. There is no central place where one can find out about many popular extensions. Atom 1.0 is an XML namespace and may contain elements or attributes from other XML namespaces.
   • RSS 2.0 does not specify the handling of relative URI references. Different feed readers implement differing heuristics for their interpretation. There is no interoperability. In practice, relative URI references cannot be used in RSS feeds.Atom 1.0 specifies use of the XML’s built-in xml:base attribute for allowing the use of relative references.
   • RSS 2.0 can be encrypted or signed like any other web content, if treated as arbitrary content. Rules for applying standard XML Encryption and XML Digital Signature on entries are included in Atom 1.0.
   • The RSS 2.0 specification includes no schema. Atom 1.0 includes a (non-normative) ISO-Standard RelaxNG schema, to support those who want to check the validity of data advertised as Atom 1.0.
2. Atom is run by the Atompub Working Group group is specified in RFC 4287 while RSS is basically run by a single genius who’s arguably very hard to work with.

   “Dave Winer has done a tremendous amount of work on RSS and invented important parts of it and deserves a huge amount of credit for getting us as far as we have. However, just looking around, I observe that there are many people and organizations who seem unable to maintain a good working relationship with Dave.” — Tim Bray (member of the W3C and co-creator of XML)

3. Google perfers Atom.

   “That is the sound of inevitability.”

The bottom line for me is that Atom seems to be more technically sound as well as more open. When you combine that with Google having chosen it as their preferred syndication protocol, that pretty much seals it for me.

I’m not going to be calling feeds “Atom feeds” — that would be lame — but I am going to dump the RSS bit. In my view Atom is clearly the winner long-term.:

[ An Information Security News Über-Feed ]

Here’s a Yahoo! Pipe that I’ve completed that allows you to check just one feed to get quick summary of infosec news. It pulls and aggregates from the following sources:

• netsec (reddit)
• linuxsecurity
• sans
• astalavista
• itsecurity
• hackinthebox
• rootsecure
• secguru
• helpnetsecurity
• honeynet

If you have any ideas for additions (or subtractions), do let me know.

[ An Information Security News Über-Feed ]

<

p style=”text-align: center”> humanproductivitylab.com

Few things are as important to geeks as how they stay informed. Knowledge is so key to us that the ability to pull it in, process it, and ensure that it’s recallable, is absolutely vital to being effective in our work.

One of the main problems we as information fetishists face is the lack of a solid, repeatable methodology for processing new input online. Too often we bounce back and forth between this site and that site, maybe check a blog or two, and then half-heartedly label the task of “reading news” as completed. This approach is not only a really poor way to stay on top of what’s new, but it’s also very anti-GTD.

The GTD Approach

For those not familiar with the GTD system, this essentially means that part of your mind will linger on activities that it doesn’t believe were performed adequately. This is actually a horrible way to go about your day; you want all your brain power focused on your current task, and being “sure” you’ve adequately absorbed today’s new information is crucial to this.

Below I touch on three main steps for accomplishing this goal. Following this simple methodology will ensure that you not only get the best information every time you start a news reading session, but it’ll also help you to actually trust that you’ve had a quality news gathering experience. This trust is what will allow you to move forward in a focused manner, with all your processor and RAM where it should be — on your current task.

Have Quality Sources

At the core of efficient news reading is having quality sources. The phrase, “garbage in – garbage out” comes to mind — meaning the better your sources are, and the tighter they’re tweaked towards the kind of news you are looking for, the more efficient your individual sessions will be.

Think aggregation. One of the coolest recent developments for people who value both time and information has been the emergence of meta-sources, or aggregated sources, of news. Two of these that I deem as absolutely crucial are listed below:

• Reddit
Reddit is an incredible collection of diverse content. The unique thing about this site is its focus on the more intellectual stories, i.e. subjects that branch into philosophy and politics as well as technology. I’ve seen many political sites that are heavier in this area, but none that ride the line between this space and technology so gracefully.
• Digg
Digg is an essential resource. Watching the front page for this site yields perhaps the hottest technology stories of the moment. It’s purely technical, so you’re not going to see it watered down with other types of content. If you’re a technical geek and this isn’t one of your sources you need to attack yourself.

Yes, most already know about Digg and Reddit, but they’re still worth mentioning. You have to learn to let go and limit your inputs; realize that the odds are good that anything you need to know will likely hit one of these two sites. Trust the system.

In addition to these meta-feeds you of course want to include a number of other specialized sources as well. These could range from personal blogs to industry-specific sites that keep you abreast of current events. For industry-oriented content you want to focus on the sources that bring the best content per unit of time. This seems fairly obvious, but being able to make the distinction between “cool”, “necessary”, and redundant is very important.

Not to pick on any particular site, but if you read a lot of different technical sources you may notice that content rarely shows up on Slashdot without first being hitting at least one other site. The question then becomes — do you need that site as well? Look for these types of sources within your list and see where you can trim them. I haven’t read Slashdot in over a year.

Choose What to Read Based on How Much Time You Have

So that brings us to the most important element of all this — deciding what to read at what time. I suggest that you organize your feeds into two to three groups based on priority. I call mine (creative, I know) primary, secondary, and tertiary.

The key here is to take a look at what you want to accomplish during your reading session, and then pick the feed group that corresponds to it. Here’s how my groups break down:

• Primary: My main aggregations + information security news (which comes from a Yahoo! Pipe I made). This is for getting up to speed quickly on the things that are necessary. Nothing extra goes in here.
• Secondary: This is where I put my main feeds that I care about but aren’t part of primary. If I have a bit more time I may look at these, but they are mostly after-hours reading.
• Tertiary: These are the feeds that I want to have the ability to monitor, but won’t check regularly. I never check these at work.

Here are my others in no particular order:

• blogs (personal and select professional blogs)
• security-blogs (information security blogs)
• security-news (additional infosec news)
• security-vulns (a myriad of vulnerability feeds)
• tracker-links (watches my friends’ linkrolls via del.icious and Google)
• tracker-books (watches my friends’ books)
• tracker-iphone (watches iPhone news via a custom Yahoo! Pipe)
• tracker-semantic (watches semantic web news via a custom Yahoo! Pipe)
• temporary (gathering bin for random feeds from people who contact me)

Basically, the names of my folders within Google Reader tell me exactly when they should be read. I can easily tell — based on what I’m doing and how much time I have — which areas to read and which to ignore. This type of input control is critical for reading news efficiently.

Have a Process

The second piece of the evolved news reading puzzle is being able to quickly rip through the sources that you’ve decided on. Too many people suffer on this step and end up mangled in a car wreck of bookmarks and open browser windows.

My technique is to use one application — Firefox. I use Google Reader for my RSS Reader, and the trick is to operate using the these three steps:

1. Open Google Reader within Firefox (it’s one of my homepage tabs) and select the items you want to read. When you’re done going through your feeds, close your Google Reader tab.
2. Starting on the leftmost tab, read each one until you’re done.
3. Consider your news session finished and return to your current task.

The beauty is in the first one — select your items and close your reader. The reason we get so discombobulated is that we’re constantly going down various tangents. We open a site, read for a bit, maybe follow a link, and before we know it three hours have passed. It’s bad form, and it leads to 1) feelings of never getting enough news, and 2) wasting time. This method solves that by subconsciously saying to yourself:

Once you’ve opened all the stories that interest you and closed Google Reader, everything you need to see today is now waiting for you in separate tabs.

It sounds simple enough, but this is precisely what your mind needs to hear in order to be able to proceed. Without hearing this you’re left with a quiet nagging in the back of the mind. “Did I see everything?” “What did I miss?” These sorts of thoughts keep you from functioning optimally. But now that you can dismiss this sort of tugging at your attention, you are free to fully immerse yourself in the current task.

Archive

The final piece of this whole absorption process is archiving what you find interesting. You do this not only so that you can maybe go back and brush up on it later, but so you can refer others to it easily when it comes up in conversation.

My solution for this is a combination of Del.icio.us – a free service that allows you to keep and share your bookmarks online — and the sharing option within Google Reader. The key thing about del.icio.us is its use of tags. When combined with Firefox Quicksearches, you can do instant URL bar searches of everything you’ve archived in the past (which for me is currently 1400+ sites). For me it’s as simple as this:

d linux firewall

This simple query put into my URL bar will search every bookmark I’ve ever put on the site that have those two tags assigned to them. Very powerful stuff.

Putting It All Together

Ok, so now that we’ve got all those steps, let’s see how it plays out:

1. Open Firefox and a Google Reader tab.
2. Go through your feeds and either read each item right in the reader or open it separately (in a browser tab).
3. When you’ve gone through all your feeds, close your Google Reader tab; you’re done with it.
4. Start from the left-most tab. Read each item, focusing on the core content and not the fluff. Remember, most items are 80% fluff.
5. If the item is a good reference or you may want to read it again or show it to others, archive it using a service like Del.icio.us.
6. When you’re done processing all the open tabs, you’re done reading news.

That’s it. It’s a process with a beginning and an end, which is a critical part of being able to break the cycle of inefficient news reading. You don’t want to drift around hoping that you read your news for the day — you want to know that you did.

The cool thing about this system is that you not only cover all your news in a repeatable, dependable way, but you also do so very quickly. This gives more time to do what’s important, namely putting all that newfound information to use.:

For those who use Feedburner to build your RSS feeds, you can now add a “Reddit This” option to your feed using this Feedflare snippet I’ve created.

http://dmiessler.com/feedflare/reddit.xml

It should be added to the official repository very soon (if it’s not already).

I’ve alluded to a major project a few times in recent months. Well, I’m now ready to talk about what it is. I apologize for the disjoined presentation; I’m a bit excited and will clean up as needed later.

Background

One of the most annoying problems that faces computer users is contact management. Most don’t have a truly organized digital address book, and even those that do suffer from contact-rot. This is where each passing day means one more mailing address has changed, someone got a new mobile number, and another person got married and has a new last name. In other words, time deteriorates the quality of your information about other people.

Many services have come and gone that tried (or are trying) to solve this problem. Most notable of these is Plaxo. Plaxo, as well as most of the other services like it, have essentially been services where you kept your updated information. The idea being that when you changed your info, Plaxo could notify the people in your address book that you had done so. At that point they could take some steps to update their information. The problem is that it’s required too much involvement with the third party service. Plaxo is, after all, a for-profit company, so it makes sense that they would want you to interact with them.

Identity Management + Semantic Web

My idea is simple: provide a free and open infrastructure upon which people can build identity-based services ranging from contact management to social interaction functionality. Focus on transparency and open standards, meaning that the exchange of informaton should be as simple as possible and should allow for infinite potential for securely sharing and manipulating the data.

Here are the two primary components:

1. Central, Server-Side Representation of People using XML I’m currently working on RDF for the main definition.
2. Open, RSS-based Client The client piece, while completely open to various implementations, will have two components. 1) Subscriptions to contacts via RSS, and 2) translation of the server’s XML to their own address book format.

Functionality

• Maintain constantly updated contact information by subscribing to your friends’ information on a central server. You stay updated because your information is not static. The information you see when you open your address book is what was last pulled from your contact’s RSS feed.
• Your contact list is constantly maintained in a neatly defined, XML-based format on the server (OPML?). To get your contacts onto any new system (including mobile devices), install any client (there will be many) that speaks both the server-side XML protocol and the local address book format.
• Link the elements within a given definition to other namespaces that carry weight within the semantic world. In other words, allow favorite bands, favorite foods, and a multitude of other attributes to be defined in such a way that associated information can be referenced (and mashed) semantically.

The Architecture

The server resides at kitmee.com (currently living in a VMware machine in San Fransisco that’s powered off) and hosts the various identity files (RDF, etc.). As an example, we’ll say we have two accounts — myself (Daniel Miessler), and my friend (Seth Kline).

We respectively live at kitmee.com/dmiessler and kitmee.com/skline. Within whatever client we’re using for the system (again, this will be any one of many available) I’ll subscribe to Seth’s address within my client that’s installed on my local system. The client works by maintaining two types of information: who you are, and who your subscriptions are (your contacts).

More On Client Functionality

The most basic client monitors the local address book for changes to my own contact information, and upon sensing changes translates the changed result into the server’s XML format and uploads it. This updates my information on the server and updates the associated RSS feed that represents me as a person.

Since people who have me in their “contact list” are actually just subscribed to my RSS feed, their respective clients (web clients, desktop clients, mobile clients) will be notified the next time they check in that I have updated my information. Their client will then update my information in their contact list (server-side) and make the associated change to the local address book on the system they are using (mobile phone, work computer, etc.).

So what we end up with is an infrastructure in which I can update my information using my own local address book, and that information will transparently be propogated (via RSS pull) to anyone who is subscribed to me using the system.

Once I have a client installed it disappears into the background. From that point on I interact only with my regular contact management application, and changes I make are propogated to my subscribers, and their changes are propogated to me.

The end result is that when I open my address book entry for Seth two years from now and dial his mobile number, I could very well be dialing a number that I never entered. He’ll still answer the phone on the other end, however, because at some point he updated HIS local address book, which updated the server, which updated MY local address book.

No extra steps. No extra hassle.

Considerations

Security is handled on the server by managing who can and cannot access your information. Obviously we don’t want just anyone to be able to pull your entire personal definition (essentially what’s now a vcard) by simply visiting a given URI. I also intend for the various elements/fields in the definition to be granularly controllable, e.g. work associates can see only your home number, while friends can see everything, etc.

Clients are the key; without them we don’t have the transparency that’s required to make the infrastructure useful. Specifically, we need the client to be able to translate between the server’s XML format and the local address book format. In later client iterations, however, I anticipate moving towards address book integration, i.e. being able to add kitmee subscriptions right into the native address book.

Final Thoughts

So that’s the project. I’m currently working with one other developer on the server side, and have not even started considering the client piece. Our development environment currently consists of a fairly stout Gentoo Linux server running in VMware. The application platform is RoR, and we’re using Subversion for version control.

I am very much interested in any feedback you may have. And if you’re interested in contributing — either via conceptual input or actual development effort — I’d love to hear from you. I will be following the comments in this thread and am also available via email.

This is an absolute must for anyone who struggles with explaining RSS to muggles.

[ Video: RSS In Plain English ]