nova-database team mailing list archive

[Monsyne Dragon <mdragon@xxxxxxxxxxxxx>]

Yah,  a meeting would be good.  I think there is some things that can be improved here, staunching some issues with the db layer that I think 'leak' into the design of the rest of Nova...

Some random thoughts to ponder on the below items: 


On Nov 2, 2011, at 3:22 PM, Vishvananda Ishaya wrote:

> Hey Guys,
> 
> It would probably be good to schedule an irc meeting to get the ball rolling on some db changes.  Based on some discussions that I've had recently, I see the following potential acition items.  Not all of these have been turned into blueprints yet:
> 
> 1) Switch to using scoped sessions in sqlalchemy (the naive version of this apparently breaks migrations)

+1 

> 2) Try using the pure pyhon mysql driver so eventlet can monkeypatch the calls (this probably requires 1)

+1

> 3) Sanitize all objects to dictionaries coming out of the db layer (the naive version of this is just wrapping all of the return values in dict())

Do we *really* want to do this?  I would suggest going to other way.  SQLAlchemy is a data mapper ORM, if we split the mapping from the models, the models can be Plain Old Python Objects, independent of the persistence layer,  that any db driver layer could accept and return.  That way, we could have *real* model objects  with *methods*, which would go a long way towards helping apply #7 below, amongst other things. 

> 4) Remove unused / stale db calls

Also good. 

> 5) Break db.api into multiple files

Yes, the db.api is getting to be unwieldily.  Random idea:  Make the api's be classes. There is alot of duplication of basic  xxxx_get(), xxxx_get_all(), etc methods. Inheritance could help. 
Second random thought:  the db methods could be instance methods of those classes.  since the context object is needed for all db calls anyway,  instances of the db-api classes could be be fetched from a method or property  on the context object itself, with the context passing itself into the constructor of the api object. 

i.e.    db.instance_get(context, value)   becomes some variant of: 

context.instance_api.get(value)  (or context.db_api('instance').get()  or context.db.instance.get()  .... you get the idea)  

> 6) Test the db layer directly (This will give us a good record of expected objects returned from the db layer)

+1e9  Really. 

> ---
> 7) Use the law of demeter for db objects instead of indirectly accessing subobjects (This implies a heavy performance penalty, so we will probably need smart caching where we joinedload objects when possible and return the cached object instead of reloading)
> 8) Implement a second db driver (zookeeper)
> 9) Split the dbs for different components into separate databases (this is a heavy change and will require code changes throughout the code)
> 
> 7-9 are definitely longer term goals, and they probably won't make it into the essex timeframe.  I think 1 through 6 are all doable in this release, and we may be able to make some progress on the others as well.
> 
> Vish
> 
> 
> -- 
> Mailing list: https://launchpad.net/~nova-database
> Post to     : nova-database@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nova-database
> More help   : https://help.launchpad.net/ListHelp

--
	Monsyne M. Dragon
	OpenStack/Nova 
	cell 210-441-0965
	work x 5014190