From: Eric S. Raymond Date: Thu, 3 Nov 2011 10:22:36 +0000 (-0400) Subject: Deal with the name change to freecode.net. X-Git-Tag: 2.3 X-Git-Url: http://thyrsus.com/gitweb/?a=commitdiff_plain;h=8a701f499a46960c5e09f3cd9f9897f3dc601a75;p=freecode-submit.git Deal with the name change to freecode.net. --- diff --git a/Makefile b/Makefile index 4adc57e..bddcfd2 100644 --- a/Makefile +++ b/Makefile @@ -1,44 +1,44 @@ -# Makefile for the freshmeat-submit tool +# Makefile for the freecode-submit tool -VERS=$(shell sed MANIFEST - (cd ..; ln -s freshmeat-submit freshmeat-submit-$(VERS)) - (cd ..; tar -czf freshmeat-submit/freshmeat-submit-$(VERS).tar.gz `cat freshmeat-submit/MANIFEST`) - (cd ..; rm freshmeat-submit-$(VERS)) +freecode-submit-$(VERS).tar.gz: $(SOURCES) freecode-submit.1 + find $(SOURCES) freecode-submit.1 -type f | sed "s:^:freecode-submit-$(VERS)/:" >MANIFEST + (cd ..; ln -s freecode-submit freecode-submit-$(VERS)) + (cd ..; tar -czf freecode-submit/freecode-submit-$(VERS).tar.gz `cat freecode-submit/MANIFEST`) + (cd ..; rm freecode-submit-$(VERS)) pychecker: @echo "Expect: Object (data) has no attribute (update)" - @ln -f freshmeat-submit freshmeat-submit.py - @-pychecker --only --limit 50 freshmeat-submit.py - @rm -f freshmeat-submit.py + @ln -f freecode-submit freecode-submit.py + @-pychecker --only --limit 50 freecode-submit.py + @rm -f freecode-submit.py clean: - rm -f *.pyc *.html freshmeat-submit.1 MANIFEST ChangeLog SHIPPER.* *~ + rm -f *.pyc *.html freecode-submit.1 MANIFEST ChangeLog SHIPPER.* *~ -dist: freshmeat-submit-$(VERS).tar.gz +dist: freecode-submit-$(VERS).tar.gz -release: freshmeat-submit-$(VERS).tar.gz freshmeat-submit.html +release: freecode-submit-$(VERS).tar.gz freecode-submit.html shipper -u -m -t; make clean diff --git a/NEWS b/NEWS index 2d5d817..808fae4 100644 --- a/NEWS +++ b/NEWS @@ -1,8 +1,12 @@ - freshmeat-submit project release news + freecode-submit project release news + +Thu Nov 03 2011 Eric S. Raymond - 2.3 + Deal with the Freshmeat to Freecode name change. + Means the name of this tool has to change. Thu Nov 24 2010 Eric S. Raymond - 2.2 Release withdrawal, previously broken, now works. - (This required a Freshmeat API change.) + (This required a site API change.) Thu Oct 21 2010 Eric S. Raymond - 2.1 Now has a query mode that can retrieve project metadata in editable form. @@ -11,7 +15,7 @@ Thu Oct 21 2010 Eric S. Raymond - 2.1 License changed to BSD. Mon Oct 17 2010 Eric S. Raymond - 2.0 - Updated to work with the JSON-based 3.0 freshmeat.net API. + Updated to work with the JSON-based 3.0 API. Major rewrite and simplification of the code. Mon Aug 2 2004 Eric S. Raymond - 1.6 diff --git a/README b/README index 0d99ce1..419d5f4 100644 --- a/README +++ b/README @@ -1,7 +1,11 @@ - freshmeat-submit + freecode-submit -freshmeat-submit is a script that supports remote submitssion of release -updates to Freshmeat via its XML-RPC interface. It is intended for use +freecode-submit is a script that supports remote submitssion of release +updates to Freecode via its XML-RPC interface. It is intended for use in project release scripts. + +(This project was formerly called freshmeat-submit. It changed its name +in 2011 when Freecode did.) + Eric S, Raymond December 2003 diff --git a/control b/control index 5bf409d..01c17ab 100644 --- a/control +++ b/control @@ -1,18 +1,18 @@ # This is not a real Debian control file, but the syntax is upward-compatible. # It's project metadata for the shipper tool -Package: freshmeat-submit +Package: freecode-submit -Description: submit release information to freshmeat.net - freshmeat-submit is a script that supports remote submission of - release updates to Freshmeat via its JSON API. It is intended for use +Description: submit release information to freecode.net + freecode-submit is a script that supports remote submission of + release updates to Freecode via its JSON API. It is intended for use in project release scripts. It reads the metadata from an RFC-2822-like message on standard input, possibly with overrides by command-line switches. -XBS-Destinations: freshmeat +XBS-Destinations: freecode -Homepage: http://www.catb.org/~esr/freshmeat-submit +Homepage: http://www.catb.org/~esr/freecode-submit XBS-Logo: gold-mega.png diff --git a/freecode-submit b/freecode-submit new file mode 100755 index 0000000..f473ed7 --- /dev/null +++ b/freecode-submit @@ -0,0 +1,405 @@ +#!/usr/bin/env python +# +# freecode-submit -- script transactions with the Freecode server + +import sys, re, urllib2, json, netrc, email.Parser, optparse + +version="2.3" + +class Update: + "Encapsulate dictionaries describing a project metadata update." + def __init__(self): + self.name = None + self.per_project = {} + self.urlassoc = [] + self.per_release = {} + def __repr__(self): + return "Update(" + repr(self.__dict__) + ")" + +# The Freecode API implementation is sensitive to the kind of HTTP request +# method you use. The general rule is: +# +# Reading records: GET +# Adding new records: POST +# Updating existing records: PUT +# Deleting existing records: DELETE +# +# From http://help.freecode.net/faqs/api-7/data-api-intro: +# 200 OK - Request was successful, the requested content is included +# 201 Created - Creation of a new resource was successful +# 401 Unauthorized - You need to provide an authentication code with this +# request +# 403 Forbidden - You don't have permission to access this URI +# 404 Not Found - The requested resource was not found +# 409 Conflict - The validation of your submitted data failed, please check +# the response body for error pointers +# 500 Server Error - The request hit a problem and was aborted, please report +# as a bug if it persists +# 503 Service Unavailable - You hit your API credit limit + +class RequestWithMethod(urllib2.Request): + "Hack for forcing the method in a request - allows PUT and DELETE" + def __init__(self, method, *args, **kwargs): + # This assignment works directly in older Python versions + self._method = method + urllib2.Request.__init__(self, *args, **kwargs) + def get_method(self): + "Deduce the HTTP method to use." + # This method works in newer Pythons (2.6.5, possibly earlier). + if self._method: + return self._method + elif self.has_data(): + return "POST" + else: + return "GET" + +class FreecodeSessionException(Exception): + def __init__(self, msg): + self.msg = msg + +class FreecodeSession: + "Encapsulate the state of a Freecode API session." + server = "http://freecode.net/" + + def __init__(self, auth=None, verbose=0, emit_enable=True): + "Initialize Freecode session credentials." + self.auth = auth + self.verbose = verbose + self.emit_enable = emit_enable + self.project = None + # If user didn't supply credentials, fetch from ~/.netrc + if not self.auth: + try: + credentials = netrc.netrc() + except netrc.NetrcParseError, e: + raise FreecodeSessionException("ill-formed .netrc: %s:%s %s" \ + % (e.filename, e.lineno, e.msg)) + except IOError, e: + raise FreecodeSessionException(("missing .netrc file %s" % \ + str(e).split()[-1])) + ret = credentials.authenticators("freecode") + if not ret: + raise FreecodeSessionException("no credentials for Freecode") + login, self.auth, password = ret + + def on_project(self, name): + "Select project by Freecode shortname." + if self.verbose: + print "Selecting project: %s" % name + self.project = name + query = FreecodeSession.server + "projects/%s.json?auth_code=%s" \ + % (self.project, self.auth) + handle = urllib2.urlopen(query) + content = json.loads(handle.read()) + self.project_data = content['project'] + #if self.verbose: + # print "Project data: %s" % self.project_data + self.permalink = self.project_data['permalink'] + self.id = self.project_data['id'] + + def edit_request(self, url, method="GET", request={}, force=False): + "Wrap a JSON object with the auth code and ship it as a request" + url = FreecodeSession.server + url + data = {"auth_code" : self.auth} + data.update(request) + data = json.dumps(data) + headers = {"Content-Type" : "application/json"} + if self.verbose: + print "Request URL:", method, url + #if self.verbose: + # print "Request headers:", headers + if self.verbose: + print "Request data:", data + if self.emit_enable or force: + req = RequestWithMethod(method=method, + url=url, + data=data, + headers=headers) + handle = urllib2.urlopen(req) + if self.verbose: + print handle.info() + content = handle.read() + if self.verbose: + print "Response:", content + return content + + def publish_release(self, data): + "Add a new release to the current project." + if self.verbose: + print "Publishing %s release: %s" % (self.project, `data`) + self.edit_request("projects/" + self.permalink + "/releases.json", + "POST", + {"release": data}) + + def withdraw_release(self, version): + "Withdraw a specified release from the current project." + if self.verbose: + print "Withdrawing %s release: %s" % (self.project, version) + releases = self.edit_request("projects/%s/releases/pending.json" \ + % self.permalink, force=True) + releases = json.loads(releases) + for release in releases: + properties = release["release"] + if properties.get("version") == version: + vid = properties["id"] + print "I see", properties + break + else: + raise FreecodeSessionException("couldn't find release %s"%version) + deletelink = "projects/%s/releases/%s.json" % (self.permalink, vid) + self.edit_request(deletelink, "DELETE", {}) + + def update_core(self, coredata): + "Update the core data for a project." + if self.verbose: + print "Core data update for %s is: %s" % (self.project, coredata) + self.edit_request("projects/" + self.permalink + ".json", + "PUT", + {"project": coredata}) + + def update_urls(self, urlassoc): + "Update URL list for a project." + if self.verbose: + print "URL list update for %s is: %s" % (self.project, urlassoc) + # First, get permalinks for all existing URLs + query = FreecodeSession.server + "projects/%s/urls.json?auth_code=%s" \ + % (self.permalink, self.auth) + handle = urllib2.urlopen(query) + content = json.loads(handle.read()) + permadict = {} + for item in content: + inner = item['url'] + permadict[inner['label']] = inner['permalink'] + # OK, now run through the update list... + for (label, url) in urlassoc: + if label in permadict: + # updating where we need to + self.edit_request("projects/%s/urls/%s.json" + % (self.permalink, permadict[label]), + "PUT", + {"url" : {"label": label, "location" : url}}) + else: + # and adding where we don't. + self.edit_request("projects/%s/urls.json" + % (self.permalink,), + "POST", + {"url" : {"label": label, "location" : url}}) + # Delete URLs that weren't in the update list + remove = set(permadict.keys()) - set(map(lambda x: x[0], urlassoc)) + for label in remove: + self.edit_request("projects/%s/urls/%s.json?auth_code=%s" + % (self.permalink, permadict[label], self.auth), + "DELETE", {}) +class FreecodeMetadataFactory: + "Factory class for producing Freecode records in JSON." + freecode_field_map = ( + ("Project", "P", "name"), # Project + ("Summary", "S", "oneliner"), # Project + ("Description", "D", "description"), # Project + ("License-List", "L", "license_list"), # Project + ("Project-Tag-List", "T", "project_tag_list"), # Project + ("Version", "v", "version"), # Release + ("Changes", "c", "changelog"), # Release + ("Hide", "x", "hidden_from_frontpage"), # Release + ("Release-Tag-List", "t", "tag_list"), # Release + # Note: Release Focus is no longer a real Freecode field. + # We remap it below. + ("Release-Focus", "r", "release_focus"), # Release + ) + # Conventional values for release focus + focus_values = ( + "Initial freecode announcement", + "Documentation", + "Code cleanup", + "Minor feature enhancements", + "Major feature enhancements", + "Minor bugfixes", + "Major bugfixes", + "Minor security fixes", + "Major security fixes", + ) + # Which attributes have project scope, all others have release scupe + projectwide = ('name', + 'description', + 'oneliner', + 'license_list', + 'project_tag_list') + + def __init__(self): + self.message_parser = email.Parser.Parser() + self.argument_parser = optparse.OptionParser( \ + usage="usage: %prog [options]") + for (msg_field, shortopt, rpc_field) in FreecodeMetadataFactory.freecode_field_map: + self.argument_parser.add_option("-" + shortopt, + "--" + msg_field.lower(), + dest=rpc_field, + help="Set the %s field"%msg_field) + self.argument_parser.add_option('-q', '--query', dest='query', + help='Query metadata for PROJECT',metavar="PROJECT") + self.argument_parser.add_option('-d', '--delete', dest='delete', + default=False, action='store_true', + help='Suppress reading fields from stdin.') + self.argument_parser.add_option('-n', '--no-stdin', dest='read', + default=True, action='store_false', + help='Suppress reading fields from stdin.') + self.argument_parser.add_option('-N', '--dryrun', dest='dryrun', + default=False, action='store_true', + help='Suppress reading fields from stdin.') + self.argument_parser.add_option('-V', '--verbose', dest='verbose', + default=False, action='store_true', + help='Enable verbose debugging.') + self.argument_parser.add_option('-?', '--showversion', dest='showversion', + default=False, action='store_true', + help='Show version and quit.') + + def header_to_field(self, hdr): + lhdr = hdr.lower().replace("-", "_") + for (alias, shortopt, field) in FreecodeMetadataFactory.freecode_field_map: + if lhdr == alias.lower().replace("-", "_").replace("/", "_"): + return field + raise FreecodeSessionException("Illegal field name %s" % hdr) + + def getMetadata(self, stream): + "Return an Update object describing project and release attributes." + data = {} + urls = {} + (options, args) = self.argument_parser.parse_args() + # Stuff from stdin if present + prior_version = data.get("version") + if not (options.query or options.showversion) and options.read: + message = self.message_parser.parse(stream) + for (key, value) in message.items(): + value = re.sub("\n +", " ", value).strip() + if key.endswith("-URL"): + key = key.replace("-", " ") + urls.update({key[:-4] : value}) + else: + if key.endswith("List"): + value = map(lambda x: x.strip(), value.split()) + data.update({self.header_to_field(key) : value}) + if not 'changelog' in data: + payload = message.get_payload().strip() + if payload: + data['changelog'] = payload + "\n" + if prior_version and data.get("version") != prior_version: + raise FreecodeSessionException("Version conflict on stdin.") + # Merge in options from the command line; + # they override what's on stdin. + controls = ('query', 'delete', 'read', 'dryrun', 'verbose', 'showversion') + prior_version = data.get("version") + for (key, value) in options.__dict__.items(): + if key not in controls and value != None: + data[key] = value + del options.__dict__[key] + if data.get("version") != prior_version and not options.delete: + raise FreecodeSessionException("Version conflict in options.") + # Hidden flag special handling + if "hidden_from_frontpage" in data: + data["hidden_from_frontpage"] = data["hidden_from_frontpage"] in ("Y", "y") + # Release-Focus field special handling. + if "release_focus" in data: + if data["release_focus"] in '123456789': + data["release_focus"] = FreecodeMetadataFactory.focus_values[int(data["release_focus"])-1] + if data["release_focus"] not in FreecodeMetadataFactory.focus_values: + raise FreecodeSessionException("Invalid release focus") + if "tag_list" not in data: + data["tag_list"] = data["release_focus" ] + else: + data["tag_list"].append(data["release_focus" ]) + del data["release_focus"] + # Now merge in the URLs, doing symbol substitution + urllist = [] + for (label, furl) in urls.items(): + for (k, v) in data.items(): + if type(v) == type(""): + furl = furl.replace('${' + k + '}', v) + urllist.append((label, furl)) + # Sort out what things go where + update = Update() + if options.showversion: + pass + elif options.query: + update.name = options.query + else: + update.name = data.pop('name') + update.urlassoc = urllist + for (k, v) in data.items(): + if k in FreecodeMetadataFactory.projectwide: + # Hack to get around a namespace collision + if k == "project_release_tag": + k = "release_tag" + update.per_project[k] = v + else: + update.per_release[k] = v + # Return this + return (options, update) + +if __name__ == "__main__": + try: + # First, gather update data from stdin and command-line switches + factory = FreecodeMetadataFactory() + (options, update) = factory.getMetadata(sys.stdin) + # Some switches shouldn't be passed to the server + query = 'query' in options.__dict__ and options.query + verbose = 'verbose' in options.__dict__ and options.verbose + delete = 'delete' in options.__dict__ and options.delete + dryrun = 'dryrun' in options.__dict__ and options.dryrun + showversion = 'showversion' in options.__dict__ and options.showversion + if showversion: + print "freecode-submit", version + raise SystemExit, 0 + # Time to ship the update. + # Establish session + session = FreecodeSession(verbose=int(verbose), emit_enable=not dryrun) + session.on_project(update.name) + if options.query: + print "Project: %s" % session.project_data["name"] + print "Summary: %s" % session.project_data["oneliner"] + print "Description: %s" % session.project_data["description"].replace("\n", "\n ").rstrip() + print "License-List: %s" % ",".join(session.project_data["license_list"]) + print "Project-Tag-List: %s" % ",".join(session.project_data["tag_list"]) + for assoc in session.project_data['approved_urls']: + #print "Querying", assoc["redirector"] + #req = RequestWithMethod(method="HEAD", + # url=assoc["redirector"], + # data={}, + # headers={}) + #handle = urllib2.urlopen(req) + #print "===" + #print handle.info() + #print "===" + print "%s-URL: %s" % (assoc["label"].replace(" ","-"), assoc["redirector"]) + if 'recent_releases' in session.project_data: + most_recent = session.project_data['recent_releases'][0] + print "Version: %s" % most_recent['version'] + print "Tag-List: %s" % ",".join(most_recent['tag_list']) + if most_recent.get('hidden_from_frontpage'): + print "Hide: Y" + else: + print "Hide: N" + print "" + print most_recent["changelog"] + else: + # OK, now actually add or delete the release. + if update.per_project: + session.update_core(update.per_project) + if update.urlassoc: + session.update_urls(update.urlassoc) + if delete: + session.withdraw_release(update.per_release['version']) + elif update.per_release and update.per_release.keys()!=["version"]: + session.publish_release(update.per_release) + except FreecodeSessionException, e: + print >>sys.stderr,"freecode-submit:", e.msg + sys.exit(1) + except urllib2.HTTPError, f: + print >>sys.stderr,"freecode-submit: HTTPError %s" % (f.code) + print >>sys.stderr,f.read() + sys.exit(1) + except urllib2.URLError, f: + print >>sys.stderr,"freecode-submit: URLError %s" % (f.reason,) + print >>sys.stderr,f.read() + sys.exit(1) + +# end + diff --git a/freecode-submit.xml b/freecode-submit.xml new file mode 100644 index 0000000..65b761c --- /dev/null +++ b/freecode-submit.xml @@ -0,0 +1,402 @@ + + + + +freecode-submit +1 +freecode-submit +Development Tools +Oct 17 2010 + + +freecode-submit +submit release announcements and project metadata updates to freecode.net + + + + + freecode-submit + -q + -V + -n + -N + -d + --showversion + data-options + data-options + + + +DESCRIPTION + +freecode-submit is a command-line tool +for submitting release announcements to freecode.net. It is intended +for batch-mode operation in project release scripts, and designed so +that its input format can easily be generated by scripts. + +freecode-submit accepts release +information from a job card (an email-message-like data block on +standard input) or from command-line flags. Values set with +command-line flags override those collected from the job card. The +accumulated data is entered into the freecode database via its +API. + +A few flags control the operation of +freecode-submit itself. + + + +-q, --query + +Query the specified project. Delivers to standard output a job +card describing the project metadata that can be edited and fed back +to this tool. The data with release rather than project scope is for +the most recent release. + + + +-d, --delete + +Delete the specified release rather than submitting it. With +this option, all data other than Project and Release are +ignored. Note: only works if the release has been submitted but not +yet passed by a moderator; if the release is already posted, invoking +this mode will throw an error. + + + +-n, --no-stdin + +Process command-line options only; don't read data from +standard input. + + + +-N, --dryrun + +Generate JSON, but don't ship any modification requests to Freecode. +Useful for debugging in combination with -V. Note, this mode may generate +requests which read data from Freecode in order to get permalink data. + + + + +-V, --verbose + +Enable verbose debugging of the transaction. This +option is mainly for developers. + + + +--showversion + +Show the version of this program and exit. + + + + +Release information is accepted from sources in the +following order: + + + +An RFC-2822-format job card, strongly resembling an ordinary +email message, on standard input. The message header fields each +specify a particular item of release information; the body (if +present) becomes the change announcement text. Within URL fields, the +string ${version} will be replaced with the contents of the Version +field. + + + +Command-line switches. + + + +Here is a table of metadata fields and their +corresponding command-line switches: + + + + + + Message field + Scope + Short switch + Long switch + Explanation + + + + + +Project: Project +-p--project + +Name of the project (freecode shortname) to operate on. You cannot +change the project-name attribute with this program; to do that, you +need to use the Freecode web interface. + + + + +Summary: Project +-S--summary + +One-line summary of the project. + + + + +Description: Project +-D--description + +Multiline description of the project. (Use RFC-822-style continuation +lines) + + + + +License-List: Project +-L--license-list + +Specify the licenses under which the project is issued. This +may be a comma-separated list. + + + + +Project-Tag-List:Release +-T--project-tag-list + +Tags to attach to this project. May be a comma-separated list. + + + + +Version: Release +-v--version + +Version string to be associated with the release. + + + + +Changes: Release +-c--changes + +The Changes field. If no Changes field or option is already +present, the changes text is taken from the body of the RFC-822 +message on standard input. + + + + +Release-Focus:Release +-r--release-focus + +Purpose of this release. See the table of release-focus types +below. + + + + +Release-Tag-List: Release +-t--tag-list + +Tags to attach to this release. May be a comma-separated list. + + + + +Hide: Release +-x--hide + +If this field is present and has the value “Y”, this +release will not be visible on the project page. + + + + + + + +Additionally, you can specify tagged URL fields for the +Freecode project record with header lines ending in "-URL:". The tag +will be the header with the "-URL:" suffix stripped off. Some examples +of how to set frequently-used URL types: + + + + + + Message field + Tag + Usage + + + + + + Website-URL: + Website + Project website home page. If you specify this, it will be used to + activate the "More Information" link and Home icon on the project page. + + + + Download-URL: + Download + Preferred download URL. If you specify this, it will be used to + activate the "Download" link anf foppy-disk on the project page. + + + + Tar/GZ-URL: + Tar/GZ + tar archive compressed with gzip + + + + Tar/BZ-URL: + Tar/BZ + tar archive compressed with bzip + + + + Changelog-URL + Changelog + project change log + + + + + + +Note that the list of URLs you ship does not merely add to the +existing list but overwrites it - that is how you delete old URLs +which are no longer valid. + +The Summary, Description, License-List, Project-Tag-List and all +URL fields, when specified, actually change the record for the +project. Only the Tag-List, Changes, Version, Release-Focus, and Hide +fields are actually stored for each version. + +Here are the release-focus types. Either the string or the +integer is acceptable. Case of the string is significant. + + + + +Initial freecode announcement1 +Documentation2 +Code cleanup3 +Minor feature enhancements4 +Major feature enhancements5 +Minor bugfixes6 +Major bugfixes7 +Minor security fixes8 +Major security fixes9 + + + + +The following elements of Freecode project metadata cannot yet +be modified using this tool: name, +translation list, programming +language list, operating system +list, approved screenshots, +and dependency records. + +Support for updating all these elements except +name is planned in future releases. + + +USAGE + +Here is an example of a release-information record that could be +fed to freecode-submit on standard input: + + +Project: doclifter +Version: 1.1 +Release-Focus: Major feature enhancements +Hide: N +Website-URL: http://www.catb.org/~esr/doclifter/ +Tar/GZ-URL: http://www.catb.org/~esr/doclifter/doclifter-${version}.tar.gz + +Fixed a bug in conditional evaluation that twadmin(8) tickled. Better +detection of pure inclusions. Better blank-section elimination. +Kleene star recognized as ... synonym in command synopses. Correct +some bugs in semantic-hint collection. Recognize Qt manual pages and +use their conventions. Better lifting of mandoc-generated +pages. Translate groff-style \[...] escapes, flag unknown ones. Can +now parse K&R style function prototypes as well as ANSI ones. This +version lifts 96% of 9829 manual pages in a full Red Hat 9 +installation with Fedora updates to *validated* XML-DocBook. + + + +WARNING +Since this program was first written, Freecode has changed its +data API significantly. While we have attempted to maintain backward +compatibility, the data model has changed in significant ways. + +There is no longer a Branch attribute. + +The old Release-Focus field is gone. As a special case, this +pragram will validate it and use it to fill in the project tags list. + +The old License field is now License-List and can take a +comma-separated list of licenses. + +All the old URL fields are gone. They're replaced by +the general mechanism for pairing URLs with labels. + +Earlier versions parsed RPM specfiles. This was poor +separation of function and has been removed. That responsibility +belong to the human or program generating the job card. + + +FILES + + + +~/.netrc + +freecode-submit expects to find your +Freecode authorization tag here. The correct form for an entry looks like +this: + + +machine freecode + account <your_api_auth_code> + + +For security, make sure your ~/.netrc file +is not world-readable; we recommend doing chmod 600 +~/.netrc. + + + + + +BUGS + +The URLs returned in a project metadata query are Freecode redirectors, +not the actual URLS supplied by previous project updates. This is due to +a limitation in the Freecode API which may be fixed in future versions +of that API. + +RETURN VALUES + +0 on success, 1 on failure. In case of failure, any error message +passed back by the Freecode API is issued to standard error. + + +AUTHORS +Eric S. Raymond esr@thyrsus.com. + + diff --git a/freshmeat-submit b/freshmeat-submit deleted file mode 100755 index 52ac4d0..0000000 --- a/freshmeat-submit +++ /dev/null @@ -1,405 +0,0 @@ -#!/usr/bin/env python -# -# freshmeat-submit -- script transactions with the Freshmeat server - -import sys, re, urllib2, json, netrc, email.Parser, optparse - -version="2.2" - -class Update: - "Encapsulate dictionaries describing a project metadata update." - def __init__(self): - self.name = None - self.per_project = {} - self.urlassoc = [] - self.per_release = {} - def __repr__(self): - return "Update(" + repr(self.__dict__) + ")" - -# The Freshmeat API implementation is sensitive to the kind of HTTP request -# method you use. The general rule is: -# -# Reading records: GET -# Adding new records: POST -# Updating existing records: PUT -# Deleting existing records: DELETE -# -# From http://help.freshmeat.net/faqs/api-7/data-api-intro: -# 200 OK - Request was successful, the requested content is included -# 201 Created - Creation of a new resource was successful -# 401 Unauthorized - You need to provide an authentication code with this -# request -# 403 Forbidden - You don't have permission to access this URI -# 404 Not Found - The requested resource was not found -# 409 Conflict - The validation of your submitted data failed, please check -# the response body for error pointers -# 500 Server Error - The request hit a problem and was aborted, please report -# as a bug if it persists -# 503 Service Unavailable - You hit your API credit limit - -class RequestWithMethod(urllib2.Request): - "Hack for forcing the method in a request - allows PUT and DELETE" - def __init__(self, method, *args, **kwargs): - # This assignment works directly in older Python versions - self._method = method - urllib2.Request.__init__(self, *args, **kwargs) - def get_method(self): - "Deduce the HTTP method to use." - # This method works in newer Pythons (2.6.5, possibly earlier). - if self._method: - return self._method - elif self.has_data(): - return "POST" - else: - return "GET" - -class FreshmeatSessionException(Exception): - def __init__(self, msg): - self.msg = msg - -class FreshmeatSession: - "Encapsulate the state of a Freshmeat API session." - server = "http://freshmeat.net/" - - def __init__(self, auth=None, verbose=0, emit_enable=True): - "Initialize Freshmeat session credentials." - self.auth = auth - self.verbose = verbose - self.emit_enable = emit_enable - self.project = None - # If user didn't supply credentials, fetch from ~/.netrc - if not self.auth: - try: - credentials = netrc.netrc() - except netrc.NetrcParseError, e: - raise FreshmeatSessionException("ill-formed .netrc: %s:%s %s" \ - % (e.filename, e.lineno, e.msg)) - except IOError, e: - raise FreshmeatSessionException(("missing .netrc file %s" % \ - str(e).split()[-1])) - ret = credentials.authenticators("freshmeat") - if not ret: - raise FreshmeatSessionException("no credentials for Freshmeat") - login, self.auth, password = ret - - def on_project(self, name): - "Select project by Freshmeat shortname." - if self.verbose: - print "Selecting project: %s" % name - self.project = name - query = FreshmeatSession.server + "projects/%s.json?auth_code=%s" \ - % (self.project, self.auth) - handle = urllib2.urlopen(query) - content = json.loads(handle.read()) - self.project_data = content['project'] - #if self.verbose: - # print "Project data: %s" % self.project_data - self.permalink = self.project_data['permalink'] - self.id = self.project_data['id'] - - def edit_request(self, url, method="GET", request={}, force=False): - "Wrap a JSON object with the auth code and ship it as a request" - url = FreshmeatSession.server + url - data = {"auth_code" : self.auth} - data.update(request) - data = json.dumps(data) - headers = {"Content-Type" : "application/json"} - if self.verbose: - print "Request URL:", method, url - #if self.verbose: - # print "Request headers:", headers - if self.verbose: - print "Request data:", data - if self.emit_enable or force: - req = RequestWithMethod(method=method, - url=url, - data=data, - headers=headers) - handle = urllib2.urlopen(req) - if self.verbose: - print handle.info() - content = handle.read() - if self.verbose: - print "Response:", content - return content - - def publish_release(self, data): - "Add a new release to the current project." - if self.verbose: - print "Publishing %s release: %s" % (self.project, `data`) - self.edit_request("projects/" + self.permalink + "/releases.json", - "POST", - {"release": data}) - - def withdraw_release(self, version): - "Withdraw a specified release from the current project." - if self.verbose: - print "Withdrawing %s release: %s" % (self.project, version) - releases = self.edit_request("projects/%s/releases/pending.json" \ - % self.permalink, force=True) - releases = json.loads(releases) - for release in releases: - properties = release["release"] - if properties.get("version") == version: - vid = properties["id"] - print "I see", properties - break - else: - raise FreshmeatSessionException("couldn't find release %s"%version) - deletelink = "projects/%s/releases/%s.json" % (self.permalink, vid) - self.edit_request(deletelink, "DELETE", {}) - - def update_core(self, coredata): - "Update the core data for a project." - if self.verbose: - print "Core data update for %s is: %s" % (self.project, coredata) - self.edit_request("projects/" + self.permalink + ".json", - "PUT", - {"project": coredata}) - - def update_urls(self, urlassoc): - "Update URL list for a project." - if self.verbose: - print "URL list update for %s is: %s" % (self.project, urlassoc) - # First, get permalinks for all existing URLs - query = FreshmeatSession.server + "projects/%s/urls.json?auth_code=%s" \ - % (self.permalink, self.auth) - handle = urllib2.urlopen(query) - content = json.loads(handle.read()) - permadict = {} - for item in content: - inner = item['url'] - permadict[inner['label']] = inner['permalink'] - # OK, now run through the update list... - for (label, url) in urlassoc: - if label in permadict: - # updating where we need to - self.edit_request("projects/%s/urls/%s.json" - % (self.permalink, permadict[label]), - "PUT", - {"url" : {"label": label, "location" : url}}) - else: - # and adding where we don't. - self.edit_request("projects/%s/urls.json" - % (self.permalink,), - "POST", - {"url" : {"label": label, "location" : url}}) - # Delete URLs that weren't in the update list - remove = set(permadict.keys()) - set(map(lambda x: x[0], urlassoc)) - for label in remove: - self.edit_request("projects/%s/urls/%s.json?auth_code=%s" - % (self.permalink, permadict[label], self.auth), - "DELETE", {}) -class FreshmeatMetadataFactory: - "Factory class for producing Freshmeat records in JSON." - freshmeat_field_map = ( - ("Project", "P", "name"), # Project - ("Summary", "S", "oneliner"), # Project - ("Description", "D", "description"), # Project - ("License-List", "L", "license_list"), # Project - ("Project-Tag-List", "T", "project_tag_list"), # Project - ("Version", "v", "version"), # Release - ("Changes", "c", "changelog"), # Release - ("Hide", "x", "hidden_from_frontpage"), # Release - ("Release-Tag-List", "t", "tag_list"), # Release - # Note: Release Focus is no longer a real Freshmeat field. - # We remap it below. - ("Release-Focus", "r", "release_focus"), # Release - ) - # Conventional values for release focus - focus_values = ( - "Initial freshmeat announcement", - "Documentation", - "Code cleanup", - "Minor feature enhancements", - "Major feature enhancements", - "Minor bugfixes", - "Major bugfixes", - "Minor security fixes", - "Major security fixes", - ) - # Which attributes have project scope, all others have release scupe - projectwide = ('name', - 'description', - 'oneliner', - 'license_list', - 'project_tag_list') - - def __init__(self): - self.message_parser = email.Parser.Parser() - self.argument_parser = optparse.OptionParser( \ - usage="usage: %prog [options]") - for (msg_field, shortopt, rpc_field) in FreshmeatMetadataFactory.freshmeat_field_map: - self.argument_parser.add_option("-" + shortopt, - "--" + msg_field.lower(), - dest=rpc_field, - help="Set the %s field"%msg_field) - self.argument_parser.add_option('-q', '--query', dest='query', - help='Query metadata for PROJECT',metavar="PROJECT") - self.argument_parser.add_option('-d', '--delete', dest='delete', - default=False, action='store_true', - help='Suppress reading fields from stdin.') - self.argument_parser.add_option('-n', '--no-stdin', dest='read', - default=True, action='store_false', - help='Suppress reading fields from stdin.') - self.argument_parser.add_option('-N', '--dryrun', dest='dryrun', - default=False, action='store_true', - help='Suppress reading fields from stdin.') - self.argument_parser.add_option('-V', '--verbose', dest='verbose', - default=False, action='store_true', - help='Enable verbose debugging.') - self.argument_parser.add_option('-?', '--showversion', dest='showversion', - default=False, action='store_true', - help='Show version and quit.') - - def header_to_field(self, hdr): - lhdr = hdr.lower().replace("-", "_") - for (alias, shortopt, field) in FreshmeatMetadataFactory.freshmeat_field_map: - if lhdr == alias.lower().replace("-", "_").replace("/", "_"): - return field - raise FreshmeatSessionException("Illegal field name %s" % hdr) - - def getMetadata(self, stream): - "Return an Update object describing project and release attributes." - data = {} - urls = {} - (options, args) = self.argument_parser.parse_args() - # Stuff from stdin if present - prior_version = data.get("version") - if not (options.query or options.showversion) and options.read: - message = self.message_parser.parse(stream) - for (key, value) in message.items(): - value = re.sub("\n +", " ", value).strip() - if key.endswith("-URL"): - key = key.replace("-", " ") - urls.update({key[:-4] : value}) - else: - if key.endswith("List"): - value = map(lambda x: x.strip(), value.split()) - data.update({self.header_to_field(key) : value}) - if not 'changelog' in data: - payload = message.get_payload().strip() - if payload: - data['changelog'] = payload + "\n" - if prior_version and data.get("version") != prior_version: - raise FreshmeatSessionException("Version conflict on stdin.") - # Merge in options from the command line; - # they override what's on stdin. - controls = ('query', 'delete', 'read', 'dryrun', 'verbose', 'showversion') - prior_version = data.get("version") - for (key, value) in options.__dict__.items(): - if key not in controls and value != None: - data[key] = value - del options.__dict__[key] - if data.get("version") != prior_version and not options.delete: - raise FreshmeatSessionException("Version conflict in options.") - # Hidden flag special handling - if "hidden_from_frontpage" in data: - data["hidden_from_frontpage"] = data["hidden_from_frontpage"] in ("Y", "y") - # Release-Focus field special handling. - if "release_focus" in data: - if data["release_focus"] in '123456789': - data["release_focus"] = FreshmeatMetadataFactory.focus_values[int(data["release_focus"])-1] - if data["release_focus"] not in FreshmeatMetadataFactory.focus_values: - raise FreshmeatSessionException("Invalid release focus") - if "tag_list" not in data: - data["tag_list"] = data["release_focus" ] - else: - data["tag_list"].append(data["release_focus" ]) - del data["release_focus"] - # Now merge in the URLs, doing symbol substitution - urllist = [] - for (label, furl) in urls.items(): - for (k, v) in data.items(): - if type(v) == type(""): - furl = furl.replace('${' + k + '}', v) - urllist.append((label, furl)) - # Sort out what things go where - update = Update() - if options.showversion: - pass - elif options.query: - update.name = options.query - else: - update.name = data.pop('name') - update.urlassoc = urllist - for (k, v) in data.items(): - if k in FreshmeatMetadataFactory.projectwide: - # Hack to get around a namespace collision - if k == "project_release_tag": - k = "release_tag" - update.per_project[k] = v - else: - update.per_release[k] = v - # Return this - return (options, update) - -if __name__ == "__main__": - try: - # First, gather update data from stdin and command-line switches - factory = FreshmeatMetadataFactory() - (options, update) = factory.getMetadata(sys.stdin) - # Some switches shouldn't be passed to the server - query = 'query' in options.__dict__ and options.query - verbose = 'verbose' in options.__dict__ and options.verbose - delete = 'delete' in options.__dict__ and options.delete - dryrun = 'dryrun' in options.__dict__ and options.dryrun - showversion = 'showversion' in options.__dict__ and options.showversion - if showversion: - print "freshmeat-submit", version - raise SystemExit, 0 - # Time to ship the update. - # Establish session - session = FreshmeatSession(verbose=int(verbose), emit_enable=not dryrun) - session.on_project(update.name) - if options.query: - print "Project: %s" % session.project_data["name"] - print "Summary: %s" % session.project_data["oneliner"] - print "Description: %s" % session.project_data["description"].replace("\n", "\n ").rstrip() - print "License-List: %s" % ",".join(session.project_data["license_list"]) - print "Project-Tag-List: %s" % ",".join(session.project_data["tag_list"]) - for assoc in session.project_data['approved_urls']: - #print "Querying", assoc["redirector"] - #req = RequestWithMethod(method="HEAD", - # url=assoc["redirector"], - # data={}, - # headers={}) - #handle = urllib2.urlopen(req) - #print "===" - #print handle.info() - #print "===" - print "%s-URL: %s" % (assoc["label"].replace(" ","-"), assoc["redirector"]) - if 'recent_releases' in session.project_data: - most_recent = session.project_data['recent_releases'][0] - print "Version: %s" % most_recent['version'] - print "Tag-List: %s" % ",".join(most_recent['tag_list']) - if most_recent.get('hidden_from_frontpage'): - print "Hide: Y" - else: - print "Hide: N" - print "" - print most_recent["changelog"] - else: - # OK, now actually add or delete the release. - if update.per_project: - session.update_core(update.per_project) - if update.urlassoc: - session.update_urls(update.urlassoc) - if delete: - session.withdraw_release(update.per_release['version']) - elif update.per_release and update.per_release.keys()!=["version"]: - session.publish_release(update.per_release) - except FreshmeatSessionException, e: - print >>sys.stderr,"freshmeat-submit:", e.msg - sys.exit(1) - except urllib2.HTTPError, f: - print >>sys.stderr,"freshmeat-submit: HTTPError %s" % (f.code) - print >>sys.stderr,f.read() - sys.exit(1) - except urllib2.URLError, f: - print >>sys.stderr,"freshmeat-submit: URLError %s" % (f.reason,) - print >>sys.stderr,f.read() - sys.exit(1) - -# end - diff --git a/freshmeat-submit.xml b/freshmeat-submit.xml deleted file mode 100644 index 2e6cc2d..0000000 --- a/freshmeat-submit.xml +++ /dev/null @@ -1,402 +0,0 @@ - - - - -freshmeat-submit -1 -freshmeat-submit -Development Tools -Oct 17 2010 - - -freshmeat-submit -submit release announcements and project metadata updates to freshmeat.net - - - - - freshmeat-submit - -q - -V - -n - -N - -d - --showversion - data-options - data-options - - - -DESCRIPTION - -freshmeat-submit is a command-line tool -for submitting release announcements to freshmeat.net. It is intended -for batch-mode operation in project release scripts, and designed so -that its input format can easily be generated by scripts. - -freshmeat-submit accepts release -information from a job card (an email-message-like data block on -standard input) or from command-line flags. Values set with -command-line flags override those collected from the job card. The -accumulated data is entered into the freshmeat database via its -API. - -A few flags control the operation of -freshmeat-submit itself. - - - --q, --query - -Query the specified project. Delivers to standard output a job -card describing the project metadata that can be edited and fed back -to this tool. The data with release rather than project scope is for -the most recent release. - - - --d, --delete - -Delete the specified release rather than submitting it. With -this option, all data other than Project and Release are -ignored. Note: only works if the release has been submitted but not -yet passed by a moderator; if the release is already posted, invoking -this mode will throw an error. - - - --n, --no-stdin - -Process command-line options only; don't read data from -standard input. - - - --N, --dryrun - -Generate JSON, but don't ship any modification requests to Freshmeat. -Useful for debugging in combination with -V. Note, this mode may generate -requests which read data from Freshmeat in order to get permalink data. - - - - --V, --verbose - -Enable verbose debugging of the transaction. This -option is mainly for developers. - - - ---showversion - -Show the version of this program and exit. - - - - -Release information is accepted from sources in the -following order: - - - -An RFC-2822-format job card, strongly resembling an ordinary -email message, on standard input. The message header fields each -specify a particular item of release information; the body (if -present) becomes the change announcement text. Within URL fields, the -string ${version} will be replaced with the contents of the Version -field. - - - -Command-line switches. - - - -Here is a table of metadata fields and their -corresponding command-line switches: - - - - - - Message field - Scope - Short switch - Long switch - Explanation - - - - - -Project: Project --p--project - -Name of the project (freshmeat shortname) to operate on. You cannot -change the project-name attribute with this program; to do that, you -need to use the Freshmeat web interface. - - - - -Summary: Project --S--summary - -One-line summary of the project. - - - - -Description: Project --D--description - -Multiline description of the project. (Use RFC-822-style continuation -lines) - - - - -License-List: Project --L--license-list - -Specify the licenses under which the project is issued. This -may be a comma-separated list. - - - - -Project-Tag-List:Release --T--project-tag-list - -Tags to attach to this project. May be a comma-separated list. - - - - -Version: Release --v--version - -Version string to be associated with the release. - - - - -Changes: Release --c--changes - -The Changes field. If no Changes field or option is already -present, the changes text is taken from the body of the RFC-822 -message on standard input. - - - - -Release-Focus:Release --r--release-focus - -Purpose of this release. See the table of release-focus types -below. - - - - -Release-Tag-List: Release --t--tag-list - -Tags to attach to this release. May be a comma-separated list. - - - - -Hide: Release --x--hide - -If this field is present and has the value “Y”, this -release will not be visible on the project page. - - - - - - - -Additionally, you can specify tagged URL fields for the -Freshmeat project record with header lines ending in "-URL:". The tag -will be the header with the "-URL:" suffix stripped off. Some examples -of how to set frequently-used URL types: - - - - - - Message field - Tag - Usage - - - - - - Website-URL: - Website - Project website home page. If you specify this, it will be used to - activate the "More Information" link and Home icon on the project page. - - - - Download-URL: - Download - Preferred download URL. If you specify this, it will be used to - activate the "Download" link anf foppy-disk on the project page. - - - - Tar/GZ-URL: - Tar/GZ - tar archive compressed with gzip - - - - Tar/BZ-URL: - Tar/BZ - tar archive compressed with bzip - - - - Changelog-URL - Changelog - project change log - - - - - - -Note that the list of URLs you ship does not merely add to the -existing list but overwrites it - that is how you delete old URLs -which are no longer valid. - -The Summary, Description, License-List, Project-Tag-List and all -URL fields, when specified, actually change the record for the -project. Only the Tag-List, Changes, Version, Release-Focus, and Hide -fields are actually stored for each version. - -Here are the release-focus types. Either the string or the -integer is acceptable. Case of the string is significant. - - - - -Initial freshmeat announcement1 -Documentation2 -Code cleanup3 -Minor feature enhancements4 -Major feature enhancements5 -Minor bugfixes6 -Major bugfixes7 -Minor security fixes8 -Major security fixes9 - - - - -The following elements of Freshmeat project metadata cannot yet -be modified using this tool: name, -translation list, programming -language list, operating system -list, approved screenshots, -and dependency records. - -Support for updating all these elements except -name is planned in future releases. - - -USAGE - -Here is an example of a release-information record that could be -fed to freshmeat-submit on standard input: - - -Project: doclifter -Version: 1.1 -Release-Focus: Major feature enhancements -Hide: N -Website-URL: http://www.catb.org/~esr/doclifter/ -Tar/GZ-URL: http://www.catb.org/~esr/doclifter/doclifter-${version}.tar.gz - -Fixed a bug in conditional evaluation that twadmin(8) tickled. Better -detection of pure inclusions. Better blank-section elimination. -Kleene star recognized as ... synonym in command synopses. Correct -some bugs in semantic-hint collection. Recognize Qt manual pages and -use their conventions. Better lifting of mandoc-generated -pages. Translate groff-style \[...] escapes, flag unknown ones. Can -now parse K&R style function prototypes as well as ANSI ones. This -version lifts 96% of 9829 manual pages in a full Red Hat 9 -installation with Fedora updates to *validated* XML-DocBook. - - - -WARNING -Since this program was first written, Freshmeat has changed its -data API significantly. While we have attempted to maintain backward -compatibility, the data model has changed in significant ways. - -There is no longer a Branch attribute. - -The old Release-Focus field is gone. As a special case, this -pragram will validate it and use it to fill in the project tags list. - -The old License field is now License-List and can take a -comma-separated list of licenses. - -All the old URL fields are gone. They're replaced by -the general mechanism for pairing URLs with labels. - -Earlier versions parsed RPM specfiles. This was poor -separation of function and has been removed. That responsibility -belong to the human or program generating the job card. - - -FILES - - - -~/.netrc - -freshmeat-submit expects to find your -Freshmeat authorization tag here. The correct form for an entry looks like -this: - - -machine freshmeat - account <your_api_auth_code> - - -For security, make sure your ~/.netrc file -is not world-readable; we recommend doing chmod 600 -~/.netrc. - - - - - -BUGS - -The URLs returned in a project metadata query are Freshmeat redirectors, -not the actual URLS supplied by previous project updates. This is due to -a limitation in the Freshmeat API which may be fixed in future versions -of that API. - -RETURN VALUES - -0 on success, 1 on failure. In case of failure, any error message -passed back by the Freshmeat API is issued to standard error. - - -AUTHORS -Eric S. Raymond esr@thyrsus.com. - -