Feature Goals for Planet 1.0

============================

* Store and use feed history

We have a huge problem at the moment, in that we only display the current

contents of the feeds we download. The old Netscape standard was 15 items

in a feed, so many people still use that. Unfortunately, on sites like

Slashdot, those 15 items move *very* quickly. So as they fall off the feed,

they fall off your Planet aggregate. That is bad, bad, bad, and we need to

solve it. Any ideas? There are potential bugs all over this. :-)

* Add multiple keyword support

I hadn't announced it yet, but I have added a simple 'keyword feeds'

feature, which allows you to subscribe to feeds and only display items that

mention your keyword in the title, summary or description. This is totally

arse-kickingly rad, and demonstrated on my personal Planet.

However, currently it only allows you to define one keyword phrase for each

feed, which kinda sucks. If there's some quick-and-dirty boolean phrase

parser, we could use it really well here, for instance:

keyword = (gnome OR kde) NOT sucks

Maybe that's overkill - we could just support multiple ORed keyword fields.

* Fix stupid UTF-8 error with current keyword support

For some reason, keyword support tweaks string encoding errors, such as:

"WARNING:root:Item wasn't in UTF-8 or ISO-8859-1, replaced all non-ASCII

characters." This is bad.

* Allow templates to use extra feed elements

See Frederic's thread about Freshmeat the other day. I don't want to

special case everything, and I don't want to stomp on item variables we

already have, but it would be nice to bring special things from feeds up

into the template data structure. Should we only do this for namespaced

elements (extensions)?

* Fix intermittent timezone detection / arithmetic bugs

From Nick Moffitt: "Right now there's something screwy in the way planet

deals with time zones, so that all you Australians end up claiming the top

of my planet while merkins I want to read end up shoved down a ways."

* Allow display normalisation to specified timezone

Some Planet admins would like their feed to be displayed in the local

timezone, instead of UTC.

* Add "sort by entry received date" option

From Nick Moffitt: "Is there a way to get planet to not do entry-date

placement at all? Like, always put new entries up in the order they were

first seen by planet? I use mine to implement the "show me stuff I haven't

already read yet" feature that pretty much all Web sites lack.

Answer from Scott J. Remnant: "Yeah, this would be actually pretty easy ...

it already has this code for entries missing times entirely, I'll add a

config option to just outright ignore feed times and sort by seen order.

I'll make it a per-feed option, maybe "sort_order = seen/time" and if you

want it global just stick it in [DEFAULT]."

* Support OPML and foaf subscriptions

This might be a bit invasive, but I want to be able to subscribe to OPML

and FOAF files, and see each feed as if it were subscribed individually.

Perhaps we can do this with a two-pass configuration scheme, first to pull

the static configs, second to go fetch and generate the dynamic configs.

The more I think about it, the less invasive it sounds. Hmm.

* Provide a 'disabled' configuration option

Make it easy to mark a feed as disabled, but still show it on the subs

list. Perhaps disabled is the wrong word. Some people want to use Planet to

do their blogroll management, but not actually show some feeds in the

aggregation. If someone can think of a better word, please tell me.

Perhaps 'hide'.

Other Feature Goals

===================

* Port to feedparser 3.0

__authors__ = [ "Scott James Remnant <scott@netsplit.com>",

"Jeff Waugh <jdub@perkypants.org>" ]

__license__ = "Python"

import os

import sys

import time

import dbhash

import ConfigParser

import planet

def usage():

print "Usage: planet-cache [options] CACHEFILE [ITEMID]..."

print

print "Examine and modify information in the Planet cache."

print

print "Channel Commands:"

print " -C, --channel Display known information on the channel"

print " -L, --list List items in the channel"

print " -K, --keys List all keys found in channel items"

print

print "Item Commands (need ITEMID):"

print " -I, --item Display known information about the item(s)"

print " -H, --hide Mark the item(s) as hidden"

print " -U, --unhide Mark the item(s) as not hidden"

print

print "Other Options:"

print " -h, --help Display this help message and exit"

sys.exit(0)

def usage_error(msg, *args):

print >>sys.stderr, msg, " ".join(args)

print >>sys.stderr, "Perhaps you need --help ?"

sys.exit(1)

def print_keys(item, title):

keys = item.keys()

keys.sort()

key_len = max([ len(k) for k in keys ])

print title + ":"

for key in keys:

if item.key_type(key) == item.DATE:

value = time.strftime(planet.TIMEFMT_ISO, item[key])

else:

value = str(item[key])

print " %-*s %s" % (key_len, key, fit_str(value, 74 - key_len))

def fit_str(string, length):

if len(string) <= length:

return string

else:

return string[:length-4] + " ..."

if __name__ == "__main__":

cache_file = None

want_ids = 0

ids = []

command = None

for arg in sys.argv[1:]:

if arg == "-h" or arg == "--help":

usage()

elif arg == "-C" or arg == "--channel":

if command is not None:

usage_error("Only one command option may be supplied")

command = "channel"

elif arg == "-L" or arg == "--list":

if command is not None:

usage_error("Only one command option may be supplied")

command = "list"

elif arg == "-K" or arg == "--keys":

if command is not None:

usage_error("Only one command option may be supplied")

command = "keys"

elif arg == "-I" or arg == "--item":

if command is not None:

usage_error("Only one command option may be supplied")

command = "item"

want_ids = 1

elif arg == "-H" or arg == "--hide":

if command is not None:

usage_error("Only one command option may be supplied")

command = "hide"

want_ids = 1

elif arg == "-U" or arg == "--unhide":

if command is not None:

usage_error("Only one command option may be supplied")

command = "unhide"

want_ids = 1

elif arg.startswith("-"):

usage_error("Unknown option:", arg)

else:

if cache_file is None:

cache_file = arg

elif want_ids:

ids.append(arg)

else:

usage_error("Unexpected extra argument:", arg)

if cache_file is None:

usage_error("Missing expected cache filename")

elif want_ids and not len(ids):

usage_error("Missing expected entry ids")

# Open the cache file directly to get the URL it represents

try:

db = dbhash.open(cache_file)

url = db["url"]

db.close()

except dbhash.bsddb._db.DBError, e:

print >>sys.stderr, cache_file + ":", e.args[1]

sys.exit(1)

except KeyError:

print >>sys.stderr, cache_file + ": Probably not a cache file"

sys.exit(1)

# Now do it the right way :-)

my_planet = planet.Planet(ConfigParser.ConfigParser())

my_planet.cache_directory = os.path.dirname(cache_file)

channel = planet.Channel(my_planet, url)

for item_id in ids:

if not channel.has_item(item_id):

print >>sys.stderr, item_id + ": Not in channel"

sys.exit(1)

# Do the user's bidding

if command == "channel":

print_keys(channel, "Channel Keys")

elif command == "item":

for item_id in ids:

item = channel.get_item(item_id)

print_keys(item, "Item Keys for %s" % item_id)

elif command == "list":

print "Items in Channel:"

for item in channel.items(hidden=1, sorted=1):

print " " + item.id

print " " + time.strftime(planet.TIMEFMT_ISO, item.date)

if hasattr(item, "title"):

print " " + fit_str(item.title, 70)

if hasattr(item, "hidden"):

print " (hidden)"

elif command == "keys":

keys = {}

for item in channel.items():

for key in item.keys():

keys[key] = 1

keys = keys.keys()

keys.sort()

print "Keys used in Channel:"

for key in keys:

print " " + key

print

print "Use --item to output values of particular items."

elif command == "hide":

for item_id in ids:

item = channel.get_item(item_id)

if hasattr(item, "hidden"):

print item_id + ": Already hidden."

else:

item.hidden = "yes"

channel.cache_write()

print "Done."

elif command == "unhide":

for item_id in ids:

item = channel.get_item(item_id)

if hasattr(item, "hidden"):

del(item.hidden)

else:

print item_id + ": Not hidden."

channel.cache_write()

print "Done."