Monday, 2010-09-20

*** oshani has quit IRC00:21
*** oshani has joined #cc00:23
CC_HudsonYippie, build fixed!01:05
CC_HudsonProject deedscraper build (40): FIXED in 4 min 17 sec: http://code.creativecommons.org/hudson/job/deedscraper/40/01:05
CC_HudsonProject cc.engine build (101): STILL FAILING in 5 min 57 sec: http://code.creativecommons.org/hudson/job/cc.engine/101/01:06
CC_HudsonProject metadata_scraper build (41): SUCCESS in 1 min 42 sec: http://code.creativecommons.org/hudson/job/metadata_scraper/41/01:07
*** akila87 has joined #cc01:52
*** akila871 has joined #cc02:12
*** akila87 has quit IRC02:15
*** pmiller has joined #cc03:21
*** akila871 has left #cc04:04
*** oshani has quit IRC04:08
*** pmiller has quit IRC04:13
*** oshani has joined #cc04:14
*** pmiller has joined #cc04:17
*** oshani has quit IRC04:41
*** pmiller has left #cc05:05
*** wormsxulla has quit IRC06:15
*** wormsxulla has joined #cc06:28
*** Kaetemi has joined #cc06:36
*** phurl has joined #cc07:00
*** phurl_ has joined #cc07:13
*** phurl has quit IRC07:15
*** phurl__ has joined #cc07:28
*** phurl_ has quit IRC07:28
*** phurl__ has quit IRC07:31
*** BjornW has joined #cc07:34
*** phurl__ has joined #cc07:36
*** phurl__ has quit IRC07:42
*** MarkDude has quit IRC08:03
*** BjornW has quit IRC08:24
*** BjornW has joined #cc08:31
*** isforinsects has quit IRC08:46
*** netAction has joined #cc09:10
netActionHello!09:11
netActionI would like to use CC for an photo shooting.09:11
netActionThe contract must include the rights of the model and some other photography specific details.09:12
netActionIs there any ready to use contract or does the CC have layers that could write such a contract?09:12
*** dAnjou has joined #cc09:45
dAnjouhi, is it possible to get images of my license with higher resolution?09:46
netActionhttp://commons.wikimedia.org/wiki/File:Cc-by_white.svg10:00
netActionHere are all icons as vector graphics.10:00
*** erlehmann has joined #cc10:23
*** juk has joined #cc11:41
jukIm c noob, here my pastebin 496958, how do I make it now instead of using hardcoded string to take from stdin?11:42
netActionFirst step is to choose the correct channel.11:43
juknetAction: what channel?11:44
dAnjou#<your programming language>-<your native language>11:45
*** juk has left #cc11:46
*** oshani has joined #cc11:46
*** akila87 has joined #cc11:52
*** oshani has quit IRC11:54
*** akila87 has left #cc11:57
*** akila871 has joined #cc11:57
*** akila871 has left #cc11:58
*** akila871 has joined #cc12:00
*** oshani has joined #cc12:32
*** tvol has joined #cc12:48
*** kreynen has joined #cc13:06
*** bassel has quit IRC13:07
*** oshani has quit IRC13:38
*** bassel has joined #cc13:42
*** DNS777 has quit IRC13:59
*** oshani has joined #cc14:05
*** DNS777 has joined #cc14:06
*** kreynen has quit IRC14:14
*** kreynen has joined #cc14:16
*** kreynen has quit IRC14:28
*** DNS777 has quit IRC14:31
*** DNS777 has joined #cc14:34
*** bassel_ has joined #cc14:39
*** bassel has quit IRC14:41
*** nkinkade has joined #cc14:42
*** nyergler has joined #cc14:54
*** fieldse has joined #cc14:54
*** ArneBab has quit IRC15:03
*** ArneBab has joined #cc15:13
*** kreynen has joined #cc15:20
*** fieldse has quit IRC15:20
*** bassel_ is now known as bassel16:10
*** phurl has joined #cc16:28
*** akozak has joined #cc16:34
*** mralex has joined #cc16:55
*** akila871 has left #cc17:13
*** phurl has quit IRC17:22
*** JED3 has joined #cc17:52
*** MarkDude has joined #cc17:58
*** MarkDude has joined #cc17:58
*** paroneayea has joined #cc18:42
paroneayeahello #cc18:52
*** bassel has quit IRC19:16
*** bassel has joined #cc19:36
paroneayeankinkade: this bug is bizarre19:39
paroneayeaat first I thought it was simple19:39
paroneayeabecause indeed, license.title()'s call in the result page of the template wasn't passing in target_lang19:39
nkinkadeTurns out to be something not so easy to find, paroneayea?19:39
paroneayeabut it turns out even when I switched it so that it does19:40
nkinkadeparoneayea: Varnish???19:40
paroneayeasomehow target_lang got turned into 'en' inside the template rendering19:40
nkinkadeOh.19:40
paroneayeaeven though outside the render, in the python view19:40
paroneayeaI can see context['target_lang'] == 'pt'19:40
paroneayeaweird stuff19:40
nkinkadeStrange.19:40
paroneayeaI wonder if ZPT overrides that somewhere and I didn't know19:40
nkinkadeparoneayea: If that really is a bug, then perhaps you should create a ticket for it?19:44
nkinkadeI didn't right away, because it wasn't 100% clear to me that it was bug in the code.19:44
CC_HudsonProject cc.engine build (102): STILL FAILING in 3 min 13 sec: http://code.creativecommons.org/hudson/job/cc.engine/102/19:49
CC_HudsonChristopher Allan Webber: license.title should be license.title(target_lang)19:49
*** bassel has quit IRC19:50
*** bassel has joined #cc19:56
nkinkadeparoneayea: The person also mentioned that the license titles on the deeds on staging weren't reflecting the license titles as he translated them in the PO file.19:59
paroneayeankinkade: yeah, I have no idea how to fix that one20:00
nkinkadeYou'll notice here: http://staging.creativecommons.org/licenses/by-nc-nd/2.5/pt/20:00
nkinkadeparoneayea: Which one?  The earlier bug?20:01
paroneayeankinkade: problem here is that we're running the license title out of the RDF now20:01
paroneayeaand the rdf files have to be "regenerated" with the different translation components20:01
paroneayeaand then after regenerated with translations, all re-merged into one big index.rdf file20:01
nkinkadeparoneayea: Ah, so this is just a staging "bug"?20:01
paroneayeankinkade: it's harder than the other staging bugs20:02
nkinkadeWhen the licenses are launched on the live site, the license titles will be correct.20:02
paroneayeankinkade: so, maybe we could automate this,20:02
nkinkadeJust to be sure, I'm also talking about something different now, and not the email I sent to webmaster@20:02
paroneayeayes, you are talking about a harder, messier bug :)20:02
paroneayeawhere those translations have to be regenerated inside the RDF20:03
paroneayeainside the *license* rdf *file*20:03
nkinkadeparoneayea: So you mean the one about the license titles not getting updated from the PO ... not about the English license names in the chooser, right?20:03
paroneayeayes.20:03
nkinkadeAh.20:03
paroneayeaso let me explain a bit better why this is a mess:20:03
nkinkadeWell, as long as it's something you are aware of and we can be assured that when some translations go live, that titles will be correctly rendered as they are in the PO file.20:04
paroneayeathey will be... if you ask me to regenerate the license titles inside the RDF :)20:05
nkinkadeparoneayea: And you have scripts that automate that?20:05
paroneayeaI have the scripts, but this can't be automated on a cronjob/hook easily20:05
nkinkade:-(20:06
*** bassel has quit IRC20:06
paroneayeabecause the regeneration *completely* regenerates the files, and then all the regenerated files have to be merged into a brand-new index.rdf20:06
paroneayeaso it's a HUGE commit difference20:06
paroneayeabloating the repository by many megabytes every time it's done20:06
paroneayeathis is because RDF does not preserve ordering.20:07
paroneayeaif we want to ever automate this, we need to figure out how to write an RDF writer/serializer that somehow tracks ordering20:07
paroneayeathat's the price/cost of the two factors of:20:08
nkinkadeWhy do we store them as RDF, again?20:08
nkinkade(That is somewhat of a rhetorical question, though not 100%)20:09
nkinkadeSeems like a pain in the ass, for not much benefit, and just a lot of headaches.20:09
paroneayeankinkade: part of "sanity" is that "the RDF should be good enough to be the database"20:09
nkinkadeYeah, but I'm rather wondering why we don't just *use* a database, then we could have scripts that could output just about any format we wanted.20:10
paroneayeathe idea was that applies to license titles as well20:10
paroneayeawell20:10
nkinkadeWho uses this RDF except CC?20:10
paroneayeankinkade: at this point rewriting the tools to NOT use the RDF  would be an insane overhaul20:10
paroneayeathis is the *only* problem where the RDF is "not good enough"20:11
paroneayeaall that data is already there20:11
nkinkadeI know.  I'm just wondering how we ever went down the RDF road in the first place.  It's a pointless question since we can't turn back now.20:11
nkinkadeparoneayea: Actually, I've seen emails where people are asking, now is RDF the only format for this data?20:11
paroneayeankinkade: the answer to that is now "yes"20:12
paroneayeathat was one of the goals of sanity20:12
paroneayeaany other formats should be generated FROM the RDF20:12
paroneayeankinkade: there are a couple of fixes to the license title problem which don't involve us scrapping RDF20:12
nkinkadeparoneayea: But so few people have experience writing code around and processing RDF.20:12
paroneayeankinkade: that's true, though what other format would we use?20:13
nkinkadeIt almost seems like we chose RDF to be "correct" at the expense of making it a huge project for ourselves.20:13
paroneayeaXML?20:13
paroneayeano, I don't think that's true20:14
paroneayeaRDF works nicely and is fairly elegant as our datastore, with the exception of20:14
paroneayea - the current license title issue (fixable)20:14
paroneayea - few people know about RDF but CC :\20:15
nkinkadeparoneayea: I'm not sure about which format is best.  At first glance I have to wonder why it must be a file format at all and not really in a database where the data is extracted into whatever format we have bothered to write a script to pull out.20:15
paroneayeankinkade: I'm not sure what benefit a database would provide20:15
paroneayeaI imagine it would create a lot *more* problems20:15
paroneayeabecause:20:15
paroneayea - we'd have to extract it to some other formats anyway20:16
nkinkadeparoneayea: Standardized tools that have huge support and are commonly understood.20:16
paroneayea - we'd have to provide a "dump" of the database regularly20:16
nkinkadeparoneayea: We have to extract it from the RDF as it is.20:16
nkinkadeThe RDF is essentially an inscrutable file for virtually any human, and most code.20:16
paroneayeankinkade: by extract it I mean "provide it in a format that non-cc people can use"20:17
paroneayeankinkade: I'm not saying I'm a fan of RDF ;)20:17
paroneayeabut I don't think a database that we'd provide (and would have to host a public accessible interface for querying) would be better20:17
paroneayeaand no other file based solution really seems much better20:17
nkinkadeparoneayea: Out of curiosity, what is the performance of the python RDF libs?  I mean, to extract the data into some usable format is some RDF query faster than, say, a MySQL query?20:18
paroneayeankinkade: I don't think performance is really that great, it's "fast enough" and then we have caching ;)20:18
paroneayeankinkade: let me put it this way20:19
*** wormsxulla has quit IRC20:19
paroneayeasay we were running it out of MySQL20:19
paroneayea - how would non-CC employees query our license database?20:19
nkinkadeHmm.  Well, anyway, I guess I'm mostly curious about this from a historical perspective ... like, how did we get here.  RDF is what it is and it's what we have and it's here to stay, apparently.20:19
nkinkadeparoneayea: Let me answer that with another question.  How does WordPress extract all that data for blog viewers?20:20
paroneayealol20:20
paroneayeankinkade: by posting it in a non-very-parsable HTML rendering?20:21
nkinkadeWriting code, in pretty much any language, to extract data out of a relational database is mostly trivial.20:21
nkinkadehaha20:21
paroneayeankinkade: I think you're right that we should provide a version that people can query20:21
paroneayeathat isn't RDF.20:22
nkinkadeBut the point is, once it's in a MySQL database, there are thousands of tools and many languages and libraries  that can be used to extract the data into whatever format, including HTML.20:22
paroneayeaI think an XML export would be good20:22
paroneayeasince every language has XML parsing tools20:22
nkinkadeYeah, XML is pretty well supported, as is SQL.20:22
nkinkadeIn any case, I'm sure there are other good reasons for RDF that I just don't know about ... ones that NRY and/or Mike could point out.20:23
paroneayeaat the very least, it makes sense having RDF as the canonical version from which "other versions" are exported because of RDFa20:23
nkinkadeparoneayea: But the RDF we have is hardly like the RDFa that gets put into our little HTML code snippets from the chooser.20:24
nkinkadeIt's only a 1-to-1 conceptually.20:24
paroneayeankinkade: it's consistent with the namespacing of CCRel and etc20:24
nkinkadeBut then again, there could conceptually be a 1-to-1 between RDFa and some database schema.20:25
nkinkadeI haven't really looked at the RDF much.  I'm wondering how much data duplication there is in those files.20:25
paroneayeankinkade: one thing I have to say about the RDF approach we have:20:26
paroneayeayou don't need to worry about "syncing" your SQL tables and everything across machines20:27
paroneayeawhere things are deployed and etc20:27
paroneayeawhich is a real PITA20:27
nkinkadeparoneayea: So how do you sync the RDF across machines?20:27
nkinkade"sync", that is.20:27
nkinkadersync?20:27
nkinkademysqldump > mysql is pretty trivial and reliable, too.20:28
paroneayeankinkade: you can package all the files in a tarball.  Or, in our case, check them out of a git repository, or download them with easy_install inside the python egg20:28
nkinkadeThough I will admit that transferring text files is simpler, without a doubt.20:28
paroneayeankinkade: yeah but not usable really as plaintext files, and you have to deal with schema consistency and etc, blehhhhh20:28
nkinkadeYeah, I guess I can see some benefit in their portability.20:28
paroneayeaa JSON-based DB like mongodb or whatever wouldn't be so bad20:28
paroneayeaat least that exports to files effortlessly20:29
nkinkadeBut here's really another point in my mind: who consumes this RDF except for CC?20:29
paroneayeankinkade: and THAT is a good, worthwhile question, I think :)20:29
paroneayeanobody really, and that's a problem20:30
nkinkadeThat someone can checkout the code and build their own egg seems of little interest to me, as nobody else is goingt o launch their own cc.engine.20:30
paroneayeankinkade: actually scratch that20:30
nkinkadeIs it really a problem?  Who else needs it?20:30
nkinkadeI can see people needing/wanting the API as a public way of finding things out about the CC licenses.20:31
nkinkadeBut I'm not sure else would need or really want RDF.20:31
paroneayeankinkade: that's probably true20:31
paroneayeaalthough they'd probably want an SQL dump even less ;)20:31
nkinkadehaha20:32
nkinkadeRight, but it would be trivial to write some code that gave the user a dropdown to select a format, and then click "Go" and have some file(s) generated based on some data in a database.20:32
nkinkadeI guess you can do that with RDF, too.20:33
nkinkadeBut it just seems more difficult.  But I do agree that text files are sort of nice ...20:33
nkinkadeJust not terribly efficient in this case, perhaps.20:33
nkinkadeOh well.  It's all fine.20:33
paroneayeankinkade: I am not really a big defender of RDF, and I don't know a lot about alternatives20:33
paroneayeabut all the alternatives don't seem better that I know20:33
paroneayeathey just have different problems20:33
nkinkadeYeah, I don't either.  And it could be true that it's about the best alternative for what we wanted all in all.20:34
nkinkadeI assume that was the rationale for choosing it in the first place. :-)20:34
paroneayeathe license.title translation mess is a problem though, and we do agree on that! %)20:34
*** wormsxulla has joined #cc20:34
nkinkadeBut there has been a side of me that has wanted to re-implement the deeds and chooser in with PHP and MySQL ... just for fun.20:35
nkinkade:-)20:35
paroneayeankinkade: you have fun with that! ;)20:35
nkinkadeNot that it would ever get used, but just as proof to myself that all this could be so much simpler than it is.20:35
paroneayeasounds like a good way to spend a vacation!20:35
nkinkadeNot really.20:35
nkinkadehaha20:35
paroneayeankinkade: I have one word as to why things are complicated: history20:36
nkinkadeThat project is not going to get beyond the fantasy stage.20:36
paroneayeaanything with a lot of history has a lot of inefficiencies20:36
paroneayeaunix has a lot of terrible things20:36
paroneayeawant to rewrite it?20:36
nkinkadeYeah.  Seems like we've got a boat load of code all to handle what is ostensibly some very simple requirements and circumstances.20:37
paroneayeaat least it is getting simpler, slowly20:37
paroneayeaI think! :)20:37
nkinkadeI'm not sure UNIX is a good comparison ... an OS is vastly complex because it does a vast amount of things.20:37
nkinkadeThe CC license chooser, and the Deeds and API aren't complicated at all from a user perspective.20:38
paroneayeankinkade: believe me, if you told me it was going to take a year to finish a rewrite on what looked like a dropdown at cc.org/choose/... ;)20:38
nkinkadehaha20:38
nkinkadeI know!20:38
nkinkadeThat's what I'm talking about.  It's a single page with a few radio buttons and a select box that generates 2 lines of HTML.20:38
nkinkadeYet we have hundreds of RDF files, thousands of lines of code and lots of time invested in producing those few lines of HTML.20:39
mralexO_o20:39
paroneayeankinkade: which is why I have a hard time explaining my job to people ;)20:40
nkinkadehaha20:40
paroneayeabut 2 lines of html with millions of combinations and tons of edge cases20:40
nkinkademillions of combinations?20:40
nkinkadeWe have like 50 jurisdictions, 6 licenses and perhaps 3 versions.20:41
paroneayeaand thousands of RDF files! ;D20:41
nkinkadehaha20:41
paroneayeawhat can I say man, it doesn't look like a complex system ever, any time I look at it from a toplevel perspective.  But it's hard to get into the guts of it and see how it could be much simpler20:42
nkinkadeI swear, a MySQL database could contain all the same information with about 3 tables, and a total size of couple kilobytes.20:42
nkinkadeI could probably create that database in a number of hours ... the just write 200 lines script that would generate all the RDF files on demand. :-)20:43
paroneayeankinkade: and then we'll end up back in pre-sanity madness :\20:44
paroneayeaxml files to generate rdf to generate sites20:44
paroneayeabut yeah20:44
nkinkadeThe site would be generated directly from the database, in fractions of a second.20:44
paroneayeamost of the "size" of those xml files come from:20:44
nkinkadeAnd the RDF only on demand, if any crazy person should actually want the RDF. ;-)20:45
paroneayea - translations of titles stored in the RDF20:45
nkinkadeBut again, this is all in my fantasy world.20:45
paroneayeabelieve me, my fantasy world also involves a lot less RDF :)20:45
paroneayeabut doesn't have really a clear alternative20:46
nkinkade"Progress today can be defined as man’s ability to complicate simplicity."20:46
nkinkadeThor Heyerdahl said that back in the 30s or 40s.20:46
nkinkade.... and believe every word of it.20:46
paroneayeankinkade: let's replace cc.org with a bunch of hand-edited html files and a couple of CGI files20:47
nkinkadeFrom his book Fatu Hiva.20:47
paroneayeawritten in bash.20:47
nkinkadehttp://donewith.info/notes/1920:47
paroneayeahaha20:48
paroneayeagreat quote20:48
paroneayeankinkade: well, we could just replace our XML-RDF files JSON-RDF! http://n2.talis.com/wiki/RDF_JSON_Specification20:51
paroneayeaat least those are queryable without a real RDF parser20:51
paroneayeanot much better I guess than RDF-XML20:51
paroneayea"everything sucks, everything is too complicated", conversation over20:52
nkinkadeSeriously.20:53
*** tvol has quit IRC20:57
*** oshani has quit IRC21:04
*** oshani has joined #cc21:07
*** BjornW has quit IRC21:24
CC_HudsonProject cc.engine build (103): STILL FAILING in 1 min 36 sec: http://code.creativecommons.org/hudson/job/cc.engine/103/21:27
CC_HudsonChristopher Allan Webber: somehow target_lang was overwritten by ZPT and I don't know how, so21:27
*** oshani has quit IRC21:31
mralexJED3: how goes zapport merging?21:34
JED3mralex: pretty good21:35
JED3zupport is up an running with all the lastest21:35
CC_HudsonProject cc.engine build (104): STILL FAILING in 1 min 46 sec: http://code.creativecommons.org/hudson/job/cc.engine/104/21:37
CC_Hudson* Christopher Allan Webber: license.title should be license.title(target_lang)21:37
CC_Hudson* Christopher Allan Webber: somehow target_lang was overwritten by ZPT and I don't know how, so21:37
*** kreynen has quit IRC21:39
*** sudiptamondal has joined #cc21:44
CC_HudsonProject cc.engine build (105): STILL FAILING in 1 min 53 sec: http://code.creativecommons.org/hudson/job/cc.engine/105/21:53
CC_HudsonChristopher Allan Webber: Accidentally killed the second half of the send_license_info_email in refactoring21:53
*** bassel has joined #cc21:53
paroneayeabeep21:54
*** nevyn has quit IRC21:55
*** sudiptamondal has quit IRC21:56
*** nevyn has joined #cc21:56
*** nevyn has quit IRC22:00
*** dAnjou has left #cc22:05
*** nevyn has joined #cc22:07
*** sudiptamondal has joined #cc22:08
*** CC_Hudson has quit IRC22:17
*** CC_Hudson has joined #cc22:21
JED3mralex: yeah, merging was pretty straightforward, albeit very cumbersome22:24
mralexah22:24
JED3mralex: hows that infographic coming along?22:25
*** CC_Hudson has quit IRC22:39
*** sudiptamondal has quit IRC22:42
*** oshani has joined #cc22:51
akozakFor some reason I find it surprising that Jena was developed by HP22:53
akozakHah, apparently so did they: "HPLabs management have decided not to continue with an active programme of Semantic Web research at HPL."22:54
*** karlcow has quit IRC22:59
*** kreynen has joined #cc23:04
*** oshani has quit IRC23:11
*** sudiptamondal has joined #cc23:11
*** sudiptamondal has left #cc23:12
*** sudiptamondal has joined #cc23:12
*** sudiptamondal has left #cc23:13
*** sudiptamondal has joined #cc23:14
*** ArneBab has quit IRC23:30
*** MarkDude has quit IRC23:30
*** ArneBab has joined #cc23:30
*** nkinkade has quit IRC23:32
*** Kaetemi has quit IRC23:34
*** sudiptamondal has left #cc23:36
*** karlcow has joined #cc23:39
*** nkinkade has joined #cc23:39
*** nyergler has quit IRC23:48
*** sudipta has joined #cc23:53
*** sudipta is now known as sudiptamondal23:53
*** sudiptamondal has quit IRC23:54
*** kreynen has quit IRC23:55

Generated by irclog2html.py 2.6 by Marius Gedminas - find it at mg.pov.lt!