[garbear]

Edit: uploaded here, http://www.google-melange.com/gsoc/propo...garrettb/1

[jmarshall]

Nice idea - I think a picture library is something that many users would be interested in.

Some questions to stimulate thought/discussion

How do you plan on making the database schema generic, whilst still being sufficiently fast? eg, given that pictures as a concrete implementation with EXIF metadata (year, resolution, gps info, focus info etc), plus perhaps tags for content or for people in the pictures (eg from a picasa library), what sort of database layout would you expect you'd end up with?

How would that change if the content provider (plugin for example) offered a different set of organisational tags?

What would the user expect to see in the UI from these, and how would you go about generating the necessary database queries?

Many users struggle with scanning videos. In most respects, you don't need an online metadata lookup for images, and at least for local images, there's really no reason at all not to scan as most info is in the files themselves. Given this, what process would the user go through to add a local source of images and scan metadata?

If the user then adds a flickr add-on, how would you see the process going of incorporating images from flickr alongside these other local images (either the users' own images, or images from other flickr users?)

Cheers,
Jonathan

[garbear]

How do you plan on making the database schema generic, whilst still being sufficiently fast? eg, given that pictures as a concrete implementation with EXIF metadata (year, resolution, gps info, focus info etc), plus perhaps tags for content or for people in the pictures (eg from a picasa library), what sort of database layout would you expect you'd end up with?

How would that change if the content provider (plugin for example) offered a different set of organisational tags?

If the user then adds a flickr add-on, how would you see the process going of incorporating images from flickr alongside these other local images (either the users' own images, or images from other flickr users?)

Going off of this data, EXIF contains a slew of metadata, some of which is worth keeping, and even less of which is worth sorting by. Would it speed things up if the non-sortable metadata was offloaded to a second table?

My idea for an extendable library is that there is data common to all photos (date, resolution, keywords, gps info, etc). Plugins may make additional information available (Picasa and Facebook add people tags for example; Flikr might add categories such as favorites). Plugin data would be stored in an separate table. When you view photos through the library, or run a global search, it will use the common data. When you view photos through a plugin, XBMC will provide the plugin with additional information from the separete table joined with the common table.

As far as the db layout, I would have to put some thought into it. (Should my GSoC application include the specifics of the db layout?) For starters, I'll turn to the MyPicsDB for inspiration. Here's a layout I drew of the db it uses:

What would the user expect to see in the UI from these, and how would you go about generating the necessary database queries?

The items shown in the UI are decided by the library (library.xbmc.pictures). As with the db layout, I'd have to think on which info should be shown when browsing the library, and which extended info should be shown when the user presses "i" on a photo. Ultimately, the useful vs. extraneous decision should be decided upon by multiple interested parties (Again, should my GSoC app include these specifics?)

Many users struggle with scanning videos. In most respects, you don't need an online metadata lookup for images, and at least for local images, there's really no reason at all not to scan as most info is in the files themselves. Given this, what process would the user go through to add a local source of images and scan metadata?

Libraries built on the new library by default only scan info from the files themselves. The base picture library uses EXIF, IPTC and XMP tags. Picture library plugins have access to additional file information. When a plugin scans an item to the library, the common data goes in the common library table and the additional info goes into the plugin's separate table.

The possibility for scraping data arrises when a picture library scraper addon is installed. For a picture library this makes no sense, but an anime library might want access to a scraper. In this case, after the user installs e.g. metadata.anime.anidb, they will have the option of scanning or scraping. Furthermore, scanning items to an anime library doesn't do a lot of good without the metadata, so at the the library's option, normal scanning can be disabled, forcing the user to scrape with an available scraper.

[jmarshall]

Nope, your GSoC application definitely shouldn't include these sorts of low-level specifics, but it's useful to be able to discuss them nonetheless when thinking through how you'd go about it, as that may lead you down a slightly different path.

Your schema is ofcourse picture-specific (makes sense given the source!). You have set grouping variables (countries, people, cities, categories, supplemental categories, collections, keywords) each of which has it's only separate table + link table. How extensible is this? Adding a new grouping variable means adding a table + link table, so your db layer needs support for this. Further, the more tables to query might mean you need more queries per-item to get details for display in the UI - depends whether it's needed in general lists ofcourse!

Another potential way to do it (which isn't quite as well normalized) is to use key/value pairs - i.e. something like (idFile, idKey, strValue) - that way adding a new grouping variable is as simple as adding to the key table and inserting the new stuff. This has disadvantages though in that you have one large table to join to, but that shouldn't be an issue if well indexed. You also lose the normalisation of the values (whether that makes sense, I'm not sure!) though.

Separate tables per-plugin is definitely one way to do extensibility as well, though how the plugin defines the data (and how that data may be 1:1, 1:many, many:1 or many:many) might need some thought (again key/value could be used though by sacrificing some normalisation).

Cheers,
Jonathan

[garbear]

Looks like I stand to learn a lot about database strategies. My apartment mate has his textbooks from database systems course, guess I got some reading to do And that's what having a good mentor is for, aye? c25bcbc caught my eye a while ago, I wonder if using URI for primary key here is the way to go.

and note to self: dont use MyPicsDB schema.

[garbear]

this is how I think of the library addons. I added a library extension-point, and the .xsd schema lets your addon.xml file look like this

Code:

<extension point="xbmc.library" name="Picture Library">
    <database name="MyPictures" version="1"/>
</extension>

I added a new abstract database alongside videos, music, et al. It's about 12 lines of code ATM. because it overrides CDatabase, you can bump the version in addon.xml and XBMC will automatically create a new database (MyPictures2), copy the data over, and hand the new data over to a python program including the previous version number (basically so python programs can have a UpdateOldVersion())

I also added <libraryaddonsdatabase> to as.xml, so that XBMC can use SQLite or MySQL -- all completely transparent to the python program



currently, you can disable libraries in the skin. I find this to be an egregious offence of separation of presentation and content. Now that libraries have a way to be managed, you should be able to hide an item from your home menu by disabling it in the addon manager. Likewise, enabling the addon again should show it in the home menu.



still in the concept stage, of course, but I think it'd make a good GSoC project and when the summer ends it'll finally let add-on authors make new libraries without reinventing the wheel every time

[RockerC]

Would this also indirectly enable library addons like this binary SpotyXBMC2 addon?

http://forum.xbmc.org/showthread.php?tid=67012

https://github.com/akezeke/spotyxbmc2

This binary Spotify addon integrates so well with XBMC's music library, it is like no other addon for XBMC!

Maybe you could take ideas from its source code for the Picasa and Flickr addons?

How does that SpotifyXBMC addon code differ from your library abstraction idea?

[garbear]

IMO it fits in better with Montellese's GSoC project, http://forum.xbmc.org/showthread.php?tid=127885. In order for it to mesh with this project, the music library would have to be re-implemented on top of my framework ( a discussion for another year..)