quickly team mailing list archive

[Jonathan Lange <jml@xxxxxxxxx>]

On Thu, Oct 29, 2009 at 3:09 AM, John Barstow <jbowtie@xxxxxxxxxxxxx> wrote:
> I'm looking at refactoring Quickly in a number of ways to get some
> test coverage, and I wanted to discuss them with the team before
> investing a whole lot more time in them.
>
> I have a toy implementation in my published testability branch where
> you can see the effects of what I'm talking about. It's not actually
> something that can be merged cleanly, but I can pretty easily create
> and publish a branch for that if my proposal makes sense.
>
> Current Issues
> ------------------
>
> Currently a command is implemented by writing a python (or other)
> script and including it a template. This has several undesirable
> properties.
>
> * Common commands such as edit or glade need to be maintained in each
> template, duplicating code.
> * Test modules cannot import individual functions or classes for
> testing (importing the script executes it...)
> * Standards for parsing arguments or reporting errors cannot be
> enforced meaningfully.
> * A number of support functions call sys.exit, limiting their
> suitablity for reuse.
> * Commands are mixed with template files, making the source a bit
> harder to navigate for no good reason.
>
> Proposed Solution
> -----------------------
>
> I'd like to rework the whole command infrastructure, following the
> example of Django (which makes it extremely easy to extend the
> manage.py command-line app).
>
> The new functionality will go into a new quickly.management namespace
> to avoid compatibility issues.
>
> Commands should be moved into a new commands directory within the
> template. Quickly will just look in this directory for a given
> template.
>
> All commands will inherit from BaseCommand. The ManagementUtility
> class will be responsible for locating, instantiating, and executing
> commands.  The existing Command class (which executes scripts) will be
> renamed to ShellCommand and inherit from BaseCommand.
>
> The basic flow is like so:
> * ManagementUtility.execute() will parse the argv using OptionParser
> and handle default options (like --version, --help and --template).
> * ManagementUtility.execute() will also handle the generic help (i.e.,
> listing available commands).
> * ManagementUtility.fetch_command() will load and instantiate the
> appropriate command (consulting the template first, then the core and
> finally the built-ins). Note that we consult the template first, so
> templates can easily override the built-ins if required.
> * BaseCommand.print_help() will consult the help, args, and
> option_list attributes to construct the help for individual commands.
> * BaseCommand.execute() will parse the argv with an OptionParser, call
> handle(*args, **kwargs) to do the actual work for a command, catch any
> exceptions and write a nicely formatted message to stderr.
>
> So to implement a command, we inherit from BaseCommand and override:
> *  the handle() function to do actual work. If there's an error, throw
> a CommandError exception instead of calling sys.exit().
> *  the help attribute to provide a short description of the command.
> *  the option_list attribute to customize the list of optparse options
> which will fed into the command's OptionParser.
> *  the args attribute to list the arguments accepted by the command
> (this is used to create the usage line in help messages, so something
> like "SRC DEST" would be appropriate).
>
> Finally, we create a collection of core commands, like "edit" and
> "glade" which can be reused across many templates, and move them to a
> quickly.management.core namespace. A template can replace these
> commands easily enough and in most cases probably won't.
>
> Benefits
> ----------
>
> Commands no longer have to worry about parsing arguments or handling
> the default options.
> Commands can be easily tested by calling handle() directly with
> appropriate test arguments. This means that test suites can also set
> up and tear down any resources in /tmp needed to test a command.
> Templates that want to reuse existing commands can do so without
> maintaining duplicate code.
> We can colorize terminal output - my testability branch prints errors
> in bold red, for example - by implementing functionality in
> BaseCommand.
>
> Testing
> ---------
>
> Once we have reworked this infrastructure, I'd like to propose a new
> core command - "quickly test". This should instantiate a TestRunner
> that locates and executes all unit tests and doctests in a quickly
> project. We should also generate a "tests" directory containing a
> basic unit test when a project is created.
>
> For quickly itself, we can reuse the test runner to run any doc tests
> or unit tests (which I will happily write!).
>
>
> OK, I think that's everything. Thoughts, anyone?
>

Hi John,

Your plan seems like a good one to me, although I've only ever skimmed
the Quickly code and have never even touched Django (which makes me a
bad person).

However, I'd strongly recommend taking a look at the way Bazaar does
it's command-line handling. It sounds similar to the approach you
describe in Django, so maybe there's not much to learn, but if you
look at Django and Bazaar then Quickly stands a fair chance of being
better than both.

  * bzrlib/command.py has the command infrastructure: a base Command
object, a base Option object

  * bzrlib/builtins.py has all of the builtin commands defined.
Plugins provided by commands live
    elsewhere, in what is still the best Python plugin system I've seen.

  * Commands are registered rather than auto-detected.

  * 'bzr selftest' tests the whole suite, and is very nice.

However, Didier seems to have some more thoughtful thoughts :)

jml