Wednesday, 2010-05-12

*** nyergler has quit IRC00:14
*** Max_- has joined #cc00:25
*** wormsxulla has quit IRC00:31
*** Max_- has left #cc00:32
*** mralex has quit IRC00:39
*** wormsxulla has joined #cc00:48
*** nkinkade has left #cc00:57
*** MarkDude has quit IRC00:57
*** erlehmann has joined #cc01:09
*** JoiIto has joined #cc01:48
*** MarkDude has joined #cc02:19
*** igorlukanin has joined #cc02:28
*** luisv has joined #cc02:40
*** luisv has quit IRC03:25
*** luisv has joined #cc03:27
*** igorlukanin has quit IRC03:30
*** Yaco has quit IRC03:51
*** luisv has quit IRC04:52
*** pyrak has quit IRC05:02
*** pyrak has joined #cc05:03
*** luisv has joined #cc05:19
*** TDJACR has quit IRC05:23
*** TDJACR has joined #cc05:23
*** luisv has quit IRC05:30
*** luisv has joined #cc05:36
*** ludde has joined #cc05:42
*** luisv has joined #cc05:56
*** luisv has quit IRC06:10
*** erlehmann has quit IRC06:18
*** erlehmann has joined #cc06:21
*** luisv has joined #cc06:25
*** erlehmann has quit IRC06:28
*** luisv has quit IRC06:33
*** jgay has quit IRC06:47
*** JED3 has quit IRC06:49
*** JoiIto has quit IRC08:34
*** JoiIto has joined #cc08:43
*** wormsxulla has quit IRC08:51
*** FHaag has joined #cc08:59
*** wormsxulla has joined #cc09:12
*** paroneayea has quit IRC09:23
*** paroneayea has joined #cc09:23
*** JoiIto has joined #cc09:31
*** user01 has joined #cc10:05
*** JoiIto has quit IRC10:56
*** JoiIto has joined #cc11:00
*** JoiIto has quit IRC11:06
*** midoubleko has joined #cc11:31
*** user01 has quit IRC11:32
*** anttil has quit IRC12:18
*** everton137 has quit IRC12:27
*** bassel has joined #cc12:27
*** anttil has joined #cc12:30
*** nkinkade has joined #cc13:39
*** paulproteus has quit IRC14:07
*** paulproteus has joined #cc14:10
*** paulproteus has quit IRC14:18
*** pyrak has quit IRC14:20
*** greg-g has quit IRC14:20
*** rafpaf has quit IRC14:21
*** greg-g has joined #cc14:23
*** rafpaf has joined #cc14:23
*** pyrak has joined #cc14:23
*** paulproteus has joined #cc14:26
*** luisv has joined #cc14:29
*** JoiIto has joined #cc14:31
*** luisv has quit IRC14:34
*** JoiIto has quit IRC14:46
*** oshani has joined #cc15:17
*** Kaetemi has joined #cc15:28
*** Kaetemi has joined #cc15:28
*** akila87 has joined #cc15:36
*** erlehmann has joined #cc15:42
* paulproteus waves, mornin'.15:46
paroneayeaheya paulproteus15:54
paulproteusAnyone want to try the DiscovereEd quickstart script I and rafpaf wrote? I think it works this time. (-:16:04
* paulproteus eyes greg-g16:04
*** jgay has joined #cc16:05
*** oshani has quit IRC16:10
*** oshani has joined #cc16:14
paulproteusparoneayea: Would you be interested in trying out our script?16:15
paroneayeapaulproteus: looking16:16
*** oshani has quit IRC16:25
paroneayeapaulproteus: it's running, working on it16:31
paulproteusAt the end, I hope you see a Firefox with "1 of 1 result(s)" for the search 'crime'.16:32
*** MarkDude has quit IRC16:32
paroneayeapaulproteus: btw, I see that before the fastcgi setup in the apache config file for creativecommons.org16:34
paroneayeathat you have commented out a mod_wsgi approach16:35
paroneayeaout of curiosity, did you try mod_wsgi and decide against it?16:35
paroneayeapaulproteus: it didn't :\16:50
paroneayeait looks like there were some things missing from my environment, possibly16:50
paroneayeaI'll email you the output from the script.16:50
paulproteuspaulproteus: Back then, mod_wsgi would segfault Apache.16:53
paulproteusThat totally sucked.16:53
paulproteuser, paroneayea16:53
*** mralex has joined #cc16:53
paroneayeapaulproteus: ah16:54
paroneayeaI tried installing cc.engine with it since I was having trouble figuring out the fastcgi stuff16:54
paroneayeait looked like it started a new instance of the application with every request16:54
paroneayeathat's stupid16:54
paroneayeamaybe I set it up wrong though.16:54
paroneayeapaulproteus: I'm re-running since I realized what went wrong... btw, the script says that if it already checked out discovered to remove it or move into the discovered directory17:02
paroneayeabut when I moved into the discovered it just checked out a discovered in my discovered17:02
paulproteusparoneayea: It should set up one instance and then keep that instance alive for a while.17:06
*** igorlukanin has joined #cc17:06
paroneayeapaulproteus: worked :)17:09
paulproteusYaaaay! (-:17:09
paroneayeaI didn't have sun-java6-jdk installed so it borked earlier17:09
*** mralex has left #cc17:12
*** akila87 has left #cc17:13
*** mralex has joined #cc17:13
*** JED3 has joined #cc17:20
*** oshani has joined #cc17:22
*** erlehmann has quit IRC17:24
*** JED3 has left #cc17:30
*** JED3 has joined #cc17:30
JED3nkinkade: ping17:30
JED3aren't you using .de as for your personal TLD?17:32
nkinkadeJED3: Yeah.17:33
JED3are you being affected by the outage?17:33
JED3nope, guess not17:34
nkinkadeWhat outage is that?17:34
JED3nkinkade: ^^17:36 german17:36
*** nyergler has joined #cc17:41
nyerglermorning, paroneayea17:41
nyerglerwhere do we stand on the deployment?17:41
paroneayeanyergler: hey18:12
paroneayearight now things are not much improved from last night18:13
paroneayeaI moved fastcgi testing over to staging.  I didn't make much more success figuring out what was wrong there18:13
paroneayeaso out of frustration I thought I'd try mod_wsgi, which I got working, but was broken in another way (seems it was spawning a new instance for every request)18:14
paroneayeaI'm kind of stumped, but then again I don't know much about fastcgi18:14
paroneayeaso I've been trying to read up to figure out more18:14
paroneayeaI can't get a useful error to come out of apache for the fastcgi error that's happening18:15
paroneayeaI'll write up an email detailing where I'm at and n18:15
paroneayeahow I'm stuck if that'd be useful18:15
nyerglerhave you tried setting LogLevel ?18:15
nyerglerthat'd be very useful, thanks18:15
*** michi_ has joined #cc18:16
*** igorlukanin has quit IRC18:20
*** igorlukanin has joined #cc18:21
*** pktck has joined #cc18:22
*** igorlukanin has joined #cc18:26
*** michi_ has quit IRC18:34
*** Seldon75 has joined #cc18:46
*** Seldon75 has left #cc18:46
*** wormsxulla has quit IRC19:02
*** wormsxulla has joined #cc19:03
*** igorlukanin has quit IRC19:07
nyerglerpaulproteus, you around?19:08
nyerglerparoneayea, were you going to send a status email? (we can also just talk on the phone during our call, but it might help with getting my head around it before hand)19:09
paroneayeanyergler: yes19:16
paroneayeaafter setting the loglevel19:16
paroneayeaI found that it showed: [Wed May 12 18:34:02 2010] [debug] arch/unix/fcgid_proc_unix.c(525): (111)Connection refused: mod_fcgid: can't connect unix domain socket: /var/lib/apache2/fcgid/sock/11108.319:17
paroneayeaI was trying to research what may have caused that, but haven't figured it out19:17
paroneayeaso I'm writing the email19:17
nyerglerok great19:17
*** jgay has quit IRC19:19
paulproteusnyergler: yo19:27
paulproteusWas afk for a bit; am back now.19:27
nyerglerpaulproteus, i sent you an email19:29
nyerglerpaulproteus, how do you spell "run the ded tests" ?19:30
paulproteuscat TESTING19:30
nyergleroops,my battery is about dead, going to go away for a second19:30
paulproteusant   -buildfile dedbuild.xml cc_test19:31
*** jgay has joined #cc19:31
*** nyergler has quit IRC19:34
*** bassel has quit IRC19:35
*** johnsu01 has quit IRC19:39
*** mattl has quit IRC19:42
*** mralex has quit IRC19:45
*** mralex has joined #cc19:48
nkinkadeparoneayea: Are you 100% sure that your application is behaving nicely with fcgid?19:52
nkinkadeI suspect that paulproteus could shed some light on your issue.19:52
paroneayeaI'm not sure.  JED3 may have just found something19:54
paroneayeaI'm looking at it now.  If that doesn't uncover it I'd gladly accept paulproteus' help :)19:54
paroneayeapaulproteus I mean19:55
*** midoubleko has quit IRC20:06
*** midoubleko has joined #cc20:07
paulproteusparoneayea: The above might be helpful. If you need further assistance, let me know.20:10
paroneayeapaulproteus: :)20:12
JED3nkinkade: are you cool with pushing the mtg back by 15?20:15
nkinkadeJED3: sure.20:16
JED3(already spoke with CW & NY)20:16
JED3nkinkade: great20:16
*** jgay has quit IRC20:37
*** nyergler has joined #cc20:37
paulproteusrehi, nyergler20:39
nyerglerhi paulproteus20:40
paulproteusant   -buildfile dedbuild.xml cc_test # fwiw20:40
nyergleryup, got it20:40
nyerglerpaulproteus, fwiw i'm converting with_tests to use the xml config, will push it, then look @ derbification20:43
*** pktck has quit IRC20:43
nyerglerpaulproteus, i merged master to with_testing, which is the right approach, right?20:45
nyerglerand then we'll merge back to master when we're done, correct?20:45
paulproteusThumbs up.20:46
nyerglerparoneayea, nkinkade: line 2?20:46
nkinkadenyergler: Yeah.20:46
nyerglerok, we're dialed in20:47
nyerglerpaulproteus, pushed20:47
nyerglerthe code on gitorious now uses the XML configuration file for db params20:48
nyerglerwill look at derby after this meeting20:48
*** jgay has joined #cc21:13
paroneayeaquestion... if I do this:
paroneayeashould it run metadata_scraper as john and ccengine as cwebber?21:19
nyerglerparoneayea, i don't think so; it appears to be virtualhost wide21:21
paroneayeaah okay21:21
paroneayeaso I guess john will have to own both of those directories ;)21:23
nyerglerparoneayea, i don't think it matters21:24
nyerglereither way21:24
nyergler(ie, if he owns them, you have to mess around, if you own them, he does)21:24
paroneayeayeah, it doesn't bother me :)21:25
paroneayeathat does seem to have been the root of things though!21:25
*** mattl has joined #cc21:26
paroneayea :D21:26
*** mattl is now known as Guest1606621:26
*** johnsu01 has joined #cc21:27
*** oshani has quit IRC21:29
* paulproteus wonders why mattl is now a Guest.21:30
paulproteusnyergler: With regard to your email:21:32
paulproteusWe could ditch the QuadStore class, and instead make TripleStore have a public TripleStore getStoreByName(String uri); method21:32
nyerglerparoneayea, awesome, looks like still a niggling perms issue (ie,
nyerglerpaulproteus, does that make you queasy at all?21:32
paulproteusHonestly, no. Should it?21:32
nyergleri sort of like the idea of collapsing the two of them, renaming it to RdfStore or something21:32
nyerglerno, i don't think it should21:33
paulproteusSure, okay.21:33
paroneayeanyergler: aha, that's logical21:33
paulproteusRdfStore is fine by me.21:33
*** oshani has joined #cc21:33
paulproteusThe SITE_CONFIG_URI will soon be made final and public.21:33
paulproteusAnd then we can stop repeating ourselves.21:33
nyerglerok, i made a branch for derby work (from with_testing)21:34
paulproteusI'll work on those changes in a branch right now, and I will merge that with your with_derby branch work when it's ready, and then push to with_testing.21:34
nyerglergoing to whack at that for 30 minutes or so, see where i get21:34
nyerglerbtw, i suppose we should clean up after ourselves branch wise as we merge them?21:35
nyerglernot super important, just thinking about making sure things are clear to others21:35
paulproteusYeah, agreed. I already deleted my local deploy_script branch.21:35
paulproteusIt gets to be a real pain unless you do it right when you should.21:35
* paulproteus erases the deploy_script branch.21:36
*** erlehmann has joined #cc21:38
*** Guest16066 has quit IRC21:43
*** johnsu01 has quit IRC21:43
*** jgay has quit IRC21:46
*** johnsu01 has joined #cc21:47
*** jgay has joined #cc21:47
paulproteusnyergler: If you can just get Derby compiling in with the classpath and push that, I can totally take it from there with regard to Derbification.21:49
*** mattl_ has joined #cc21:49
nyerglerpaulproteus, ok; just trying to make your tests pass right now21:51
nyerglerpaulproteus, is setup and teardown per test method or per test class?21:52
paulproteusIt's run once per method.21:52
nyerglerie, will teardown be called after every test?21:52
nyerglerhrm, weird21:52
paulproteusI think so, anyway. Emotionally, I always doubt myself on this point, but logically I feel certain about it.21:52
paulproteusThe Funny Manual for JUnit is canonical on it.21:53
paroneayeaAlias /ccengine-fcgi /var/www/
paroneayeaRewriteRule ^/(license|licenses|choose|characteristic|publicdomain)/(.*)$1/$2 [L,P]22:07
paroneayea vs
nyerglerparoneayea, are you asking for feedback?22:09
paroneayeanyergler: yes :)22:09
nyergleri'm not sure why its 403, but why I think that causes it to do a sub-request, when you really just care about /ccengine-fcgi22:11
nyergler(i assume you're doing this just to expose the few paths on the FCGI?22:11
paroneayeayou're right, no sense in the http:// probably22:13
paroneayea72.14.184.151 - - [12/May/2010:22:14:11 +0000] [][rid#cee808/initial] (1) go-ahead with proxy request proxy: [OK]22:14
nyerglerparoneayea, that actually does make some sense that'd you get a 403 --22:15
paroneayeafrom the rewrite log it looks like it *should* work..22:15
nyerglerassuming it's not configured to allow proxy requests22:15
nyergler(which is the default)22:15
paroneayeaI see.22:15
paulproteusnyergler: I want to issue a query against the TripleStore for a Curator whose getUrl() would return the value "X".22:21
paulproteusWhat's the idiomatic way to do that in this codebase?22:21
nyerglerpaulproteus, not sure; i think there's probably an idiomatic way to do it with the jenabean loader22:21
paroneayeanyergler: <- I'm clearly doing something wrong here, but what I'm not sure...22:22
paroneayeaif I can get that wrapped up I can have staging set up right before signing out in a few22:22
paroneayeathen it should be pretty easy to get things live tomorrow, I think22:23
nyerglerparoneayea, i don't think i was very clear: i think proxying only impacts http://xxx rewrites, not straight-up path rewrites22:23
nyergler(although in this instance you are denying everything before you allow everything, so nothing gets through)22:23
paroneayeaheh :)22:23
nyerglerparoneayea, it does't work w/o the proxy block?22:23
nyergleras is?22:23
paroneayeaseems not: ...although it looks less wrong22:25
paroneayeaso like:22:25
paroneayeaRewriteRule ^/(license|licenses|choose|characteristic|publicdomain)/(.*) /ccengine-fcgi/$1/$2 [L]22:25
nyerglerparoneayea, that looks right to me,22:28
nyerglerbut i'm splitting my attention right now, so may be missing something obvious22:28
paroneayeanyergler: np.... I gotta go.. I'll keep looking into it tomorrow morning22:29
nyerglerpaulproteus, i'm a little stuck22:31
paulproteusnyergler: Fire away.22:31
nyerglerseems like i'm getting left-over data between tests in the database22:31
nyerglergoing to fire up eclipse, see if it's at all useful22:31
nyerglerpaulproteus, actually, it's very clear that i'm getting leftover data22:32
paulproteusIs this on some revision I've already touched, or some patchset on top of that?22:32
paulproteusThe tests' built-in assertions should check against that; is that how you're spotting it?22:32
nyerglerpaulproteus, yes22:33
paulproteusOkay, that's good at least.22:34
nyergler(that's how i'm spotting it -- expects 0, gets 1)22:34
* paulproteus nods.22:34
paulproteusThis is with Derby? I imagine you're not deleting the Derby databases, or something?22:34
paulproteusI can clean that up, so long as you have Derby loading.22:34
nyerglerpaulproteus, yes, with derby22:35
nyergleri'm blowing away the db directory22:35
nyerglerwhere they're being created22:35
paulproteusAnd your Derby class is re-opening the files?22:36
paulproteusThat is, you're re-connecting to the database between test runs?22:36
nyerglerpaulproteus, using the dbmanager22:38
nyergler(in quadstore)22:38
nyerglerpaulproteus, it's not at all clear to me whether that caches/holds open connections22:39
nyerglerthat could prevent it from really "going away"22:39
nyerglertempted to just push this work, try to retool triple store to use the modern jena db interface22:39
nyerglerso we can pass in our own connection instead of having it handle that part, too22:40
paulproteusI don't even remember this DBManager you speak of.22:42
paulproteusI suggest you just push it to a branch and tell me to fix it.22:42
paulproteus"But of course, the consultant would say that."22:43
nyerglerpaulproteus, ok22:43
nyerglerpaulproteus, i'm going to push my branch momentarily22:45
nyerglerand then make a new one for the jena db layer22:45
nyergleri'll ping you at some point re: merging22:46
nyerglerer, actually i'll assume you'll handle merging at some point down the line :)22:46
paulproteusEven better.22:47
nyerglerpaulproteus, pushed derbification22:49
paulproteusJust saw that on the gitorious site (-:22:49
nyerglerpaulproteus, does with_testing include an updated jena?22:50
paulproteusI think so, yeah.22:51
paulproteusYeah, because we added NG4J.22:51
nyerglerpaulproteus, can i make a counter proposal to derbification?22:52
paulproteusFire away!22:52
nyergleruse Jena's native binary repr for the stores22:52
nyergler(er, ignore the hash portion)22:52
paulproteusnyergler: Sure, that's fine by me.22:52
nyerglerpaulproteus, ok, i'm going to try that and see what i come up with22:52
paulproteusOkay. Will you base that off with_derbification?22:53
nyerglerpaulproteus, no, off of with_testing, I think, as derbification will be vestigial if it works22:55
paulproteusOkay with me.22:55
paulproteusnyergler: In that case, you might like to base off my branch, one sec22:55
nyerglerpaulproteus, ok22:56
paulproteusCheck out the with_nicer_names branch.22:56
paulproteusIf it seems sane to you, treat it as the new with_testing.22:56
nyerglergreat, thanks22:58
*** oshani has quit IRC23:01
*** oshani has joined #cc23:04
nyerglerpaulproteus, ping23:09
nyergleri may have been over enthusiastic23:09
nyerglerTDB (the native format) is single reader/writer23:10
nyerglerthis seems... sub-optimal23:10
paulproteus(Is Derby?)23:10
nyerglerAFAIK Derby can have multiple connections, but maybe i'm insane23:10
paulproteusMore realistically, is that a problem?23:10
paulproteusI think Derby probably doesn't have this problem, I agree. But, like, so?23:10
nyerglersingle reader probably is -- we look up things like curator info when rendering web views23:10
paulproteusOh, drat. Right.23:11
nyerglerthat, and the crawler will be shoving data in, so that's multi-writer23:11
nyerglergoing to go look at derby a little closer23:11
nyerglerpaulproteus, so embedded derby is single connection, like tdb :/23:16
*** pyrak has quit IRC23:17
*** pyrak has joined #cc23:17
* paulproteus bites his fingernails.23:17
paulproteusI mean, we can just lock the directory and have tragic concurrency issues.23:18
nyerglertrue, looking at a couple other things right now (inc derby client/server, and, uh, ng4j :) )23:18
paulproteusHowever, that totally sucks.23:18
paulproteusOh, wait a sec.23:18
paulproteusI remember now.23:18
paulproteusWe can have multiple MySQL databases, too (-;23:18
nyerglerso carry on :)23:18
paulproteusSweet. (-:23:19
paulproteusBTW, if there's a way to use table prefixes like we're using databases, that'd be a fine solution for the long run.23:19
paulproteusTableNamePrefix   String   jena_   The common prefix for all Jena table names in the database.23:22
paulproteusSo how about we use table prefixes with the same MySQL DB, and run with that until the end of time?23:22
nyerglerpaulproteus, seems like an ok idea23:24
nyerglerpaulproteus, we're still using the old, deprecated db layer23:24
nyerglerwould like to move over to SDB eventually23:24
nyergler(er, soon ?)23:24
paulproteusRight, I saw that.23:24
paulproteusWhat benefit does it give us?23:24
*** FHaag has left #cc23:24
nyerglerit's supported? :)23:25
nyerglerand it lets us handle the connections directly with JDBC23:25
nyerglerinstead of having its own DBConnection wrapper23:25
nyerglerit's also supposed to be faster, not that big a deal for us23:25
paulproteusI propose we go as far as we can without thinking about it, then.23:26
nyerglerfair enough23:26
nyerglerso at this point you're going to keep moving with multi-dbs or prefixes (if possible), and i'm done for the moment, right?23:27
paulproteusWhat do you think of the "new me"? (-:23:27
paulproteusYeah, I think so.23:27
nyerglernot sure what to think ;)23:27
*** MarkDude has joined #cc23:29
paulproteusnyergler: Okay, I'm outta here for tonight!23:30
nyerglerpaulproteus, ok23:31
*** wormsxulla has quit IRC23:35
*** midoubleko has quit IRC23:41
*** midoubleko has joined #cc23:41
*** nyergler has quit IRC23:42
*** nyergler has joined #cc23:44
nyerglernkinkade, if you're around can you disable nagios for the staging license chooser?23:45
nkinkadenyergler: I'm here.23:45
nyergleri think things are a little borked23:45
nkinkadeparoneayea: Will you just notify when the chooser on staging is working so that I can re-enable the alert?23:45
nyerglerparoneayea is off for the day, I think23:45
nyergleri need to run, thanks for disabling that23:46
*** nyergler has quit IRC23:46
*** wormsxulla has joined #cc23:54
*** ludde has quit IRC23:58

Generated by 2.6 by Marius Gedminas - find it at!