| *** ArneBab is now known as ArneBab_away | 00:15 | |
| *** akila87 has joined #cc | 00:19 | |
| *** mralex has quit IRC | 00:24 | |
| *** JED3 has quit IRC | 00:26 | |
| *** akozak has quit IRC | 00:38 | |
| CC_Hudson | Project cc.license build (18): STILL FAILING in 2 min 6 sec: http://code.creativecommons.org/hudson/job/cc.license/18/ | 00:50 |
|---|---|---|
| CC_Hudson | * Christopher Allan Webber: Use string.template for doing mapping, because apparently mapping in | 00:50 |
| CC_Hudson | * Christopher Allan Webber: Most of the expected formatter output added, though not used in tests yet | 00:50 |
| CC_Hudson | Project deedscraper build (11): STILL FAILING in 1 min 35 sec: http://code.creativecommons.org/hudson/job/deedscraper/11/ | 01:02 |
| CC_Hudson | Project cc.engine build (21): STILL FAILING in 2 min 11 sec: http://code.creativecommons.org/hudson/job/cc.engine/21/ | 01:03 |
| CC_Hudson | Project metadata_scraper build (13): STILL FAILING in 58 sec: http://code.creativecommons.org/hudson/job/metadata_scraper/13/ | 01:03 |
| CC_Hudson | Project cc.license build (19): STILL FAILING in 1 min 10 sec: http://code.creativecommons.org/hudson/job/cc.license/19/ | 01:04 |
| CC_Hudson | Christopher Allan Webber: Moving accidental quotes and parens | 01:04 |
| *** nyergler has quit IRC | 01:15 | |
| *** paulproteus has quit IRC | 01:57 | |
| *** greg-g has quit IRC | 01:57 | |
| *** dithyramble has quit IRC | 01:57 | |
| *** pyrak has quit IRC | 01:59 | |
| *** akila87 has left #cc | 03:23 | |
| *** erlehmann has joined #cc | 03:31 | |
| *** ArneBab_away is now known as ArneBab | 03:42 | |
| *** MarkDudeva has joined #cc | 04:41 | |
| *** MarkDudeva is now known as MarkDude | 04:41 | |
| *** MarkDude has joined #cc | 04:42 | |
| *** MarkDude has quit IRC | 04:50 | |
| *** MarkDude has joined #cc | 04:54 | |
| *** pmiller has joined #cc | 05:12 | |
| *** Andre_Re1 has joined #cc | 05:29 | |
| Andre_Re1 | Hello there, just a quick question | 05:29 |
| Andre_Re1 | Does CC-BY-SA actually mean, that Ihave to redistribute a derivated work under CC-BY-SA or can I also redistribute the derivate as CC-BY? | 05:31 |
| *** Odemia has joined #cc | 05:32 | |
| *** erlehmann has quit IRC | 05:49 | |
| *** pmiller has quit IRC | 05:51 | |
| *** Odemia has quit IRC | 05:52 | |
| *** JoiIto has quit IRC | 06:04 | |
| *** Kaetemi has joined #cc | 06:40 | |
| *** tantek has quit IRC | 06:44 | |
| *** MarkDude has quit IRC | 06:58 | |
| *** thisisparker has quit IRC | 07:07 | |
| *** MarkDude has joined #cc | 07:09 | |
| *** tantek has joined #cc | 07:16 | |
| *** Andre_Re1 has left #cc | 07:27 | |
| *** pmiller has joined #cc | 07:53 | |
| *** thisisparker has joined #cc | 08:02 | |
| *** tantek has quit IRC | 08:08 | |
| *** tantek has joined #cc | 08:08 | |
| *** bassel has quit IRC | 08:14 | |
| *** Odemia has joined #cc | 08:58 | |
| *** Odemia has quit IRC | 09:15 | |
| *** Odemia has joined #cc | 09:19 | |
| *** akila87 has joined #cc | 09:27 | |
| *** tantek has quit IRC | 09:35 | |
| *** bassel has joined #cc | 09:51 | |
| *** pmiller has left #cc | 09:56 | |
| *** tantek has joined #cc | 10:26 | |
| *** tantek has joined #cc | 10:41 | |
| *** akila87 has left #cc | 10:43 | |
| *** tantek has quit IRC | 10:52 | |
| *** tantek has joined #cc | 10:54 | |
| *** akila87 has joined #cc | 11:20 | |
| *** Odemia has quit IRC | 11:26 | |
| *** karlcow has quit IRC | 11:46 | |
| *** karlcow has joined #cc | 11:55 | |
| *** bassel has quit IRC | 12:07 | |
| *** oshani has joined #cc | 12:12 | |
| *** tantek_ has joined #cc | 12:31 | |
| *** tantek has quit IRC | 12:31 | |
| *** tantek_ is now known as tantek | 12:31 | |
| *** akila87 has left #cc | 12:38 | |
| *** akila87 has joined #cc | 12:50 | |
| *** tantek has quit IRC | 13:00 | |
| *** EzOne has joined #cc | 13:11 | |
| *** EzOne has left #cc | 13:11 | |
| *** akila87 has quit IRC | 13:38 | |
| *** paulproteus has joined #cc | 14:02 | |
| *** paulproteus_ has joined #cc | 14:08 | |
| *** dithyramble has joined #cc | 14:08 | |
| *** pyrak has joined #cc | 14:08 | |
| *** paulproteus has quit IRC | 14:12 | |
| *** tantek has joined #cc | 14:13 | |
| *** tantek has quit IRC | 14:15 | |
| *** greg-g has joined #cc | 14:17 | |
| *** nkinkade has joined #cc | 14:40 | |
| alsk___ | anttil, i think SA means it's CC-BY-SA ? | 14:46 |
| alsk___ | andre_re1.. | 14:46 |
| paroneayea | nkinkade: is i.creativecommons.org being really slow or something? | 15:06 |
| paroneayea | I keep notice that on identi.ca it seems to be hanging on loading images from there | 15:07 |
| nkinkade | paroneayea: a8 has been a little slow lately, and I'm not sure what to attribute it to. | 15:08 |
| paroneayea | hurm | 15:09 |
| nkinkade | In my mind it has coincided with the the staging branch of the API being moved to dev. | 15:09 |
| *** tantek has joined #cc | 15:14 | |
| paroneayea | whoa | 15:15 |
| paroneayea | just saw the nagios alerts, looks like cc.org/apps are down | 15:16 |
| paroneayea | uhhh | 15:18 |
| paroneayea | nkinkade: wtf? | 15:18 |
| paroneayea | http://creativecommmons.org/apps/deed | 15:18 |
| paroneayea | oh | 15:18 |
| paroneayea | hahaha | 15:18 |
| paroneayea | 3 ms | 15:18 |
| paroneayea | scared myself for a minute there | 15:19 |
| paroneayea | thought we got hijacked | 15:19 |
| nkinkade | I got the message that /apps/deed was down earlier this morning. | 15:21 |
| paroneayea | yeah | 15:21 |
| nkinkade | Honestly, I'd like to move i.CC.org to it's own server. | 15:21 |
| nkinkade | I don't trust the API anymore ... even if the API isn't the problem. | 15:21 |
| nkinkade | I experienced those intermittent problems with the API timing out when viewing the /international/xx/ pages. That still hasn't been explained. | 15:22 |
| *** marucs has quit IRC | 15:22 | |
| paroneayea | yeah, I also see calls to api.cc.org slow at times too | 15:23 |
| *** SiDi has joined #cc | 15:42 | |
| SiDi | Hey there. Does anyone know where i may find a docbook version of cc-by-sa-3.0 please? | 15:42 |
| *** akila87 has joined #cc | 15:42 | |
| greg-g | paroneayea: feature request ^^^ "add ability to export license legal code in PDF, LaTeX, Plain Text, and Docbook." :P | 15:45 |
| paroneayea | greg-g: it does seem to be a common request | 15:46 |
| paroneayea | part of the problem is that the legalcode HTML is not generated by the engine | 15:46 |
| greg-g | oh :/ | 15:46 |
| greg-g | sad | 15:46 |
| paroneayea | it's hand-written by our translators | 15:46 |
| paroneayea | so that the html version of the legalcode is canonical and never changes | 15:46 |
| greg-g | huh | 15:46 |
| greg-g | oh well | 15:47 |
| paroneayea | but! | 15:47 |
| greg-g | that would be a big understanding then, no? | 15:47 |
| paroneayea | we could probably create PDF and plaintext of the core 6 licenses + cc0 maybe | 15:47 |
| paroneayea | all the ported ones though I think that's unlikely to ever happen | 15:48 |
| SiDi | paroneayea: i think it'd be great to have them maintained under a format from which it can be exported | 15:49 |
| SiDi | with docbook you'd get PDF/LaTeX/HTML under hand already | 15:49 |
| SiDi | cause now, i'm so lazy to do the CC 3.0 DocBook by myself that i'll license my doc under the 2.5 instead :D | 15:49 |
| SiDi | (i suppose CC bysa 2.5 is not compatible with GNU FDL?) | 15:49 |
| greg-g | if by compatible you mean interoperable, you are correct | 15:51 |
| SiDi | greg-g: so i could switch from cc 2.5 to FDL? | 15:56 |
| SiDi | (see how lazy i am: i choose a license based on whether there is a .xml docbook file available) | 15:57 |
| *** akila87 has left #cc | 16:09 | |
| *** tantek has quit IRC | 16:10 | |
| greg-g | SiDi: you *could* but I wouldn't recommend it :/ | 16:13 |
| SiDi | greg-g: why? | 16:13 |
| greg-g | because more and more work is being produced under CC:BY-SA instead of the FDL | 16:14 |
| paroneayea | SiDi: if you choose BY-SA, your work will be compatible with a greater number of works | 16:15 |
| paroneayea | ie, stuff on wikipedia, etc | 16:15 |
| SiDi | okey | 16:16 |
| *** JoiIto has joined #cc | 16:26 | |
| *** akila87 has joined #cc | 16:32 | |
| *** thisisparker has quit IRC | 16:37 | |
| *** tantek has joined #cc | 16:39 | |
| *** tantek has quit IRC | 16:42 | |
| *** MarkDude has quit IRC | 16:42 | |
| *** JED3 has joined #cc | 16:46 | |
| *** Endiannes has joined #cc | 17:02 | |
| *** karlushi has quit IRC | 17:02 | |
| Endiannes | I'm using a .mp3 file licences with cc 2.0, am I allowed to convert it to .ogg without modifying anything else? | 17:02 |
| *** papy|away is now known as papyromancer | 17:05 | |
| *** JED3 has quit IRC | 17:06 | |
| nkinkade | paroneayea: Hey ... is the machinery in place for commits to Transifex.net to show up automatically on staging? | 17:17 |
| nkinkade | That is, is it all up and working to your knowledge? | 17:18 |
| paroneayea | nkinkade: no :) | 17:18 |
| paroneayea | it automatically makes the tarball | 17:18 |
| paroneayea | but doesn't do a buildout and restart | 17:18 |
| paroneayea | apache | 17:18 |
| nkinkade | paroneayea: What is left to be done for that to work? | 17:18 |
| nkinkade | I think that is something we really need to address sooner than later now that we have officially moved to Transifex.net | 17:19 |
| paroneayea | nkinkade: not much. I could add to the cronjob so that it buildouts and restarts apache after it makes the egg release | 17:19 |
| nkinkade | paroneayea: Was that how we decided to tackle it, with a cronjob rather that post-receive? | 17:19 |
| nkinkade | Just trying to remember. | 17:19 |
| paroneayea | well, it seems sensible to use the cronjob now | 17:19 |
| paroneayea | since we're making the egg via cronjob. | 17:20 |
| nkinkade | paroneayea: How often does it run? | 17:20 |
| paroneayea | every 15 mins | 17:20 |
| paroneayea | would that be sufficient? | 17:20 |
| nkinkade | paroneayea: Every 15 mins seems fine to me. | 17:20 |
| nkinkade | Ideally it would all happen immediately via post-receive. | 17:20 |
| nkinkade | Is John on vacation? | 17:21 |
| paroneayea | no he's running the conference call currently being run | 17:21 |
| paroneayea | I think. | 17:22 |
| *** mralex has joined #cc | 17:27 | |
| nkinkade | Ooops. I'm not on that call. | 17:28 |
| nkinkade | Forgot. | 17:28 |
| *** JED3 has joined #cc | 17:29 | |
| nkinkade | paroneayea: how much time would it take for you to finish the cron job to have it update staging? | 17:30 |
| nkinkade | Is it really just a couple lines in a script? | 17:31 |
| paroneayea | nkinkade: yeah, just a couple of lines | 17:31 |
| nkinkade | paroneayea: Would you be willing to finish that today? :-) | 17:31 |
| paroneayea | nkinkade: sure :) | 17:31 |
| nkinkade | I'll write to CC Sweden to tell them that it should be updating by later today, then. | 17:31 |
| paroneayea | kk | 17:34 |
| *** notb has joined #cc | 17:38 | |
| nkinkade | JED3: It seems that /apps/deed and /apps/triples are down ... have been since earlier today. | 17:38 |
| *** alsk___ has quit IRC | 17:40 | |
| *** notb_ has joined #cc | 17:43 | |
| JED3 | nkinkade: thank you, how do i change the permissions set by the log roller? | 17:43 |
| nkinkade | JED3: In /etc/logrotate.d/metadata-scraper | 17:44 |
| *** notb has quit IRC | 17:45 | |
| JED3 | nkinkade: perfect thank you | 17:45 |
| paroneayea | nkinkade: done | 17:51 |
| nkinkade | paroneayea: Thanks! | 17:51 |
| paroneayea | yurp | 17:51 |
| *** Endiannes has quit IRC | 18:01 | |
| *** alsk___ has joined #cc | 18:05 | |
| *** notb_ has quit IRC | 18:08 | |
| *** thisisparker has joined #cc | 18:14 | |
| *** SiDi has quit IRC | 18:35 | |
| *** MarkDude has joined #cc | 18:48 | |
| *** alsk___ has quit IRC | 18:51 | |
| paroneayea | JED3: do you know if there's a way to tell buildout, "Just update this one package"? | 18:54 |
| JED3 | paroneayea: rm the egg and buildout, thats my solution normally | 18:54 |
| JED3 | and give the non-newest flag so the others arent updated | 18:55 |
| JED3 | -N i believe | 18:55 |
| paroneayea | JED3: maybe I should do that so as to save the server the extra load of running buildout every 15 minutes | 18:55 |
| *** MarkDude has quit IRC | 18:55 | |
| paroneayea | JED3: oh, will that work with tarballs? | 18:57 |
| paroneayea | or do you have to untar-and-do-something | 18:57 |
| *** erlehmann has joined #cc | 18:58 | |
| CC_Hudson | Project cc.i18n build (41): SUCCESS in 37 sec: http://code.creativecommons.org/hudson/job/cc.i18n/41/ | 19:22 |
| CC_Hudson | Transifex robot: l10n: Updated Georgian (ka) translation to 3% | 19:22 |
| *** ArneBab is now known as ArneBab_away | 19:29 | |
| *** click170 has quit IRC | 19:32 | |
| *** MarkDude has joined #cc | 19:33 | |
| *** akila87 has quit IRC | 19:49 | |
| *** tantek has joined #cc | 19:56 | |
| CC_Hudson | Project cc.license build (20): STILL FAILING in 1 min 55 sec: http://code.creativecommons.org/hudson/job/cc.license/20/ | 20:05 |
| CC_Hudson | * Christopher Allan Webber: String test for pdmark worktitle in place | 20:05 |
| CC_Hudson | * Christopher Allan Webber: Tests for most of these pdmark formatter possibilities, but test_cc0 failing | 20:05 |
| CC_Hudson | * Christopher Allan Webber: Forgot to switch alt text from Public Domain Mark -> CC0 on that alt tag | 20:05 |
| CC_Hudson | * Christopher Allan Webber: We don't put the dct namespace in the <p> tag when we don't use it | 20:05 |
| JED3 | paroneayea: sorry went ark | 20:06 |
| JED3 | afk | 20:06 |
| paroneayea | JED3: np | 20:06 |
| JED3 | will it work with tarballs? yes it should | 20:06 |
| paroneayea | JED3: didn't seem to, I tested :\ | 20:07 |
| JED3 | hmm, can you describe what you are trying to do? | 20:07 |
| JED3 | updating an cc.i18n dependency? | 20:07 |
| paroneayea | cp latest cc.i18n*.tar.gz from basket/ over to eggs/ | 20:08 |
| paroneayea | yeah | 20:08 |
| paroneayea | that way I don't have to run ./bin/buildout | 20:08 |
| JED3 | ohh okay | 20:08 |
| paroneayea | which seems to work just fine | 20:08 |
| paroneayea | I just suspect it'll eat up more CPU than necessary | 20:08 |
| JED3 | really that works? won't you start getting version conflicts? | 20:08 |
| *** tantek_ has joined #cc | 20:09 | |
| paroneayea | JED3: do you have a better solution? :( | 20:09 |
| paroneayea | yeah, it's fragile. Things could blow up eventually and bring down staging | 20:10 |
| paroneayea | but using buildout I don't know of a solution | 20:10 |
| JED3 | whats wrong with... | 20:10 |
| JED3 | rm -fr eggs/cc.i18n....egg/ ; bin/buildout -N ? | 20:10 |
| paroneayea | JED3: it won't do anything with an egg that's not untarred and setup.py develop'ed | 20:11 |
| paroneayea | which you can't do manually with buildout | 20:11 |
| paroneayea | brb | 20:12 |
| JED3 | paroneayea: what checkout is this for? | 20:12 |
| paroneayea | JED3: sanity on staging | 20:12 |
| *** tantek has quit IRC | 20:12 | |
| JED3 | paroneayea: setuptools with fetch the egg from basket and untar itself won't it? | 20:13 |
| paroneayea | JED3: yeah! If you don't do -N :) | 20:13 |
| paroneayea | oic what you mean | 20:13 |
| paroneayea | ohhhh | 20:13 |
| paroneayea | yeah | 20:13 |
| paroneayea | that would work | 20:13 |
| *** tantek_ has quit IRC | 20:13 | |
| JED3 | are you sure? if you delete the egg first it should work | 20:13 |
| JED3 | yeah | 20:13 |
| paroneayea | gotcha | 20:13 |
| paroneayea | yeah I'll try that | 20:13 |
| paroneayea | thx | 20:13 |
| JED3 | np | 20:14 |
| paroneayea | slight chance of it erroring out the server | 20:14 |
| paroneayea | in case some file is not cached or something weird | 20:14 |
| paroneayea | don't think that would happen though | 20:14 |
| paroneayea | (in the brief period when the egg is gone) | 20:14 |
| JED3 | yeah, thats true, you could always sleep() in the shell script | 20:15 |
| *** thisisparker has quit IRC | 20:17 | |
| CC_Hudson | Project cc.license build (21): STILL FAILING in 1 min 17 sec: http://code.creativecommons.org/hudson/job/cc.license/21/ | 20:19 |
| CC_Hudson | * Christopher Allan Webber: Escapefest test for pdmark formatter | 20:19 |
| CC_Hudson | * Christopher Allan Webber: Removed some unnecessary whitespace | 20:19 |
| *** tantek has joined #cc | 20:19 | |
| *** tantek has quit IRC | 20:24 | |
| paroneayea | nkinkade: I forget, is nyergler doing things today? | 20:27 |
| paroneayea | things that are incompatible with a weekly tech check-in | 20:27 |
| nkinkade | paroneayea: Not sure. I haven't seen him online today. | 20:27 |
| paroneayea | JED3: you know? | 20:27 |
| mralex | he's in DC | 20:27 |
| nkinkade | That would explain it. | 20:27 |
| JED3 | paroneayea: yeah i dont think we'll be having our call | 20:29 |
| paroneayea | kk | 20:29 |
| *** MarkDude has quit IRC | 20:29 | |
| *** tantek has joined #cc | 20:30 | |
| *** oshani has quit IRC | 20:34 | |
| *** tantek has quit IRC | 20:35 | |
| *** thisisparker has joined #cc | 20:35 | |
| *** erlehmann has quit IRC | 20:39 | |
| *** MarkDude has joined #cc | 20:40 | |
| *** tantek has joined #cc | 20:41 | |
| *** tantek has quit IRC | 20:45 | |
| *** tantek has joined #cc | 20:51 | |
| *** thisisparker has quit IRC | 20:54 | |
| *** tantek has quit IRC | 20:56 | |
| *** thisisparker has joined #cc | 21:00 | |
| *** tantek has joined #cc | 21:02 | |
| *** MarkDude has quit IRC | 21:04 | |
| *** tantek has quit IRC | 21:06 | |
| nkinkade | JED3: This URL seems to be loading quite slowly. My last run was 10s. On one run a minute ago it timed out. | 21:07 |
| nkinkade | http://api.creativecommons.org/rest/dev/support/jurisdictions.js | 21:07 |
| nkinkade | I just have this impression that the API has got really slow lately ... mostly just anecdotal evidence. | 21:08 |
| JED3 | wow really, there have been some serious fluctuations in the api performance | 21:08 |
| *** thisisparker has quit IRC | 21:08 | |
| nkinkade | Yeah, I'm wondering what it could all be about. Maybe it's something else causing the API to be slow? | 21:08 |
| JED3 | hmm, you know one thing that isn't there that should be, is varnish caching some of the calls that are basically static | 21:09 |
| JED3 | this jurisidictions.js call is one | 21:09 |
| JED3 | which gets clobbered by requests from cc.org | 21:10 |
| nkinkade | JED3: Yeah. I'm really noticing slowdowns on just about any /international/xx/ page. | 21:11 |
| nkinkade | Every page load there requests jurisdictions.js. In fact, I think every page load at CC(!) makes a call to that script. | 21:11 |
| JED3 | hmm yeah...i wonder if i should use nry's cache middleware | 21:12 |
| JED3 | yup | 21:12 |
| JED3 | or we could configure varnish to handle it | 21:12 |
| JED3 | which is probably easier | 21:12 |
| JED3 | its a8 thats slow for me | 21:12 |
| JED3 | just opening the docs for the api, is about to time out on me | 21:13 |
| *** tantek has joined #cc | 21:13 | |
| JED3 | which isn't served by the api process | 21:13 |
| nkinkade | JED3: But if you look at resource usage on a8, things look okay. | 21:13 |
| JED3 | http://api.creativecommons.org/docs/ nkinkade does this load quickly for you? | 21:13 |
| nkinkade | Yeah, and this morning images were coming back slow for i.CC.org. | 21:14 |
| JED3 | yeah, i know, the usages look fine | 21:14 |
| nkinkade | JED3: It took nearly 10s to load doctool.js | 21:14 |
| JED3 | nkinkade: http://api.creativecommons.org/docs/ does this load fine for you? | 21:14 |
| JED3 | it just timed out on me | 21:14 |
| nkinkade | JED3: Yeah it loaded for me, but it took nearly 10s. | 21:15 |
| nkinkade | JED3: the requests *do* hit varnish, but they don't get cached because the requests come with cookies. | 21:15 |
| JED3 | oh really? | 21:16 |
| nkinkade | And Varnish doesn't cache requests that contain cookies ... at least not by default. | 21:16 |
| nkinkade | However, api.CC.org doesn't set any cookies that I know of, or that my browser tells me about. | 21:16 |
| nkinkade | It sends them because of cookies set via creativecommons.org. | 21:16 |
| nkinkade | JED3: Wouldn't it be safe to pretty much cache every single response from the API? | 21:17 |
| JED3 | are you sure varnish is caching the rest/dev|staing/1.5/etc requests? | 21:17 |
| nkinkade | I mean, things change in the API at a glacial pace. | 21:17 |
| *** tantek has quit IRC | 21:17 | |
| nkinkade | JED3: Varnish is not caching that stuff, though the request does pass through Varnish. | 21:17 |
| nkinkade | And in any case, Varnish's default timeout is 180s, I think. And no data from the API is going to change even every 3 months, much less every 3 minutes. | 21:18 |
| nkinkade | JED3: Should I tell Varnish to start caching all API requests? | 21:18 |
| JED3 | nkinkade: thats true, but some of the api calls aren't really cacheable due to allowed parameters | 21:19 |
| nkinkade | JED3: Hmm. So Varnish won't take into consideration the query string? | 21:19 |
| nkinkade | That could be. | 21:19 |
| nkinkade | I need to check. | 21:19 |
| JED3 | nkinkade: well even if it did it might cause some cache bloat | 21:20 |
| JED3 | parameterized requests are unlikely to be called numerous times by different clients | 21:21 |
| nkinkade | How many different unique URLs are hit on any given day at the API? | 21:21 |
| nkinkade | JED3: Right, but I think the largest consumer of the API are other CC sites, right? | 21:21 |
| nkinkade | Like the license chooser and the perhaps the deeds. | 21:21 |
| JED3 | nkinkade: yes, the jurisdiction calls should definitely be cached | 21:22 |
| nkinkade | JED3: Who uses the API except for CC? At least that we know of? | 21:22 |
| *** tantek has joined #cc | 21:23 | |
| JED3 | well the wplicense does, jswidget, a few other platforms | 21:24 |
| JED3 | but probably 95% us | 21:24 |
| nkinkade | JED3: I was just looking at the API Apache logs and it seems that there have been around 60,000 requests today, from around 6,500 unique IP address, only 3766 of which requests were from a5. That sounds odd to me. | 21:27 |
| nkinkade | That would indicate that a mere 5% of the requests are coming from a5. | 21:27 |
| nkinkade | At least for today. | 21:27 |
| JED3 | wow, thats strange, because i'm awking the logs right now all i'm seeing are from a5 | 21:28 |
| *** tantek has quit IRC | 21:28 | |
| JED3 | there must be a burst in there somewhere then | 21:28 |
| JED3 | because right now, its all a5 | 21:28 |
| *** thisisparker has joined #cc | 21:28 | |
| *** tantek has joined #cc | 21:34 | |
| JED3 | nkinkade: yeah i think we're our own worst enemy here, are you sure those metrics you mentioned were accurate? | 21:37 |
| JED3 | look at the file "api-clients.log" in my home dir on a8 | 21:38 |
| *** tantek has quit IRC | 21:39 | |
| *** JoiIto has quit IRC | 21:40 | |
| nkinkade | JED3: I was looking at a5:/var/log/apache/api.creativecommons.org/access.log | 21:41 |
| nkinkade | Is that the wrong log? | 21:41 |
| *** thisisparker has quit IRC | 21:41 | |
| nkinkade | What does that log log? :-) | 21:41 |
| JED3 | nkinkade: yes, api is a8 :) | 21:42 |
| JED3 | nkinkade: beats me! | 21:42 |
| nkinkade | JED3: Right, I meant a8: | 21:43 |
| nkinkade | JED3: Where did api-clients.log come from? | 21:44 |
| JED3 | nkinkade: this... | 21:44 |
| JED3 | sudo cat /var/log/apache2/api.creativecommons.org/access.log | awk '{print $11}' | grep -v "creativecommons.org" > ~/api-clients.log | 21:44 |
| *** tantek has joined #cc | 21:45 | |
| JED3 | thats only looking at the referrers though | 21:45 |
| *** MarkDude has joined #cc | 21:48 | |
| nkinkade | JED3: Sorry, I was looking at the source address, not the referrer. | 21:49 |
| nkinkade | For some reason I felt that the source would be a5. | 21:49 |
| JED3 | nkinkade: i just updated that log file with source ip & referer | 21:49 |
| nkinkade | I thought the API calls happened behind the scenes. | 21:49 |
| *** tantek has quit IRC | 21:50 | |
| nkinkade | That is like User -> a5 -> api -> a5 -> User | 21:50 |
| nkinkade | Or something vaguely like that. | 21:50 |
| nkinkade | Is it purely javascript that accesses the API, then? | 21:50 |
| JED3 | yes, its client side | 21:51 |
| JED3 | its <script src="api.cc.org...."> which is requested by a users browser | 21:51 |
| *** tantek has joined #cc | 21:55 | |
| nkinkade | JED3: So, out of some 60,000 requests, there were only 494 unique request URLs. | 21:58 |
| nkinkade | I'm not really worried about cache bloat on the side of Varnish over ~500 distinct URIs. Beyond that, most of those requests return tiny amounts of text, which don't take up much space. | 21:59 |
| *** tantek has quit IRC | 22:00 | |
| JED3 | nkinkade: very true, lets cache it all then | 22:00 |
| nkinkade | JED3: I'll turn on caching via Varnish right now, and let's just see if we notice any difference right away ... more snappiness. | 22:01 |
| JED3 | nkinkade: ok | 22:01 |
| nkinkade | JED3: My god ... jurisdictions.js is now loading in .117ms | 22:06 |
| *** tantek has joined #cc | 22:06 | |
| JED3 | nkinkade: woot! | 22:06 |
| nkinkade | And /docs/ is loading ridiculously fast too. | 22:06 |
| nkinkade | ... at least for me. | 22:06 |
| JED3 | nkinkade: awesome, me too | 22:06 |
| JED3 | i didnt realize we werent already caching the js related calls | 22:07 |
| nkinkade | JED3: I think we just sped up page loads for lots of CC pages by a factor of 2 or 3. | 22:21 |
| nkinkade | http://creativecommons.org/international/cl/ | 22:21 |
| CC_Hudson | Project cc.i18n build (42): SUCCESS in 32 sec: http://code.creativecommons.org/hudson/job/cc.i18n/42/ | 22:21 |
| CC_Hudson | Transifex robot: l10n: Updated Russian (ru) translation to 99% | 22:21 |
| JED3 | wow good point, this uses the api server side doesnt it? | 22:22 |
| nkinkade | That URL used to consistently take many seconds to load, sometimes up to 10s. Now after the initial load and subsequent refreshes it load in about 1s. | 22:22 |
| JED3 | via a php script or something right? | 22:22 |
| nkinkade | JED3: It hits the API twice .. once for the jurisdiction list in the header behind the "International" link, and again behind the scenes to fetch that list of current licenses. | 22:22 |
| nkinkade | The one fetched by the new API call you created. | 22:23 |
| JED3 | nkinkade: must snappier i see | 22:23 |
| JED3 | nkinkade: ahh, will varnish cache resources identified by the query strings args? | 22:26 |
| *** Kaetemi has quit IRC | 22:27 | |
| *** Kaetemi has joined #cc | 22:28 | |
| nkinkade | JED3: I believe so. | 22:30 |
| JED3 | nkinkade: and varnish will cache requests for get and post? | 22:30 |
| nkinkade | I turned caching off for now. | 22:30 |
| nkinkade | JED3: I presume so. | 22:30 |
| nkinkade | Though we can make that distinction in the Varnish config file. | 22:31 |
| nkinkade | That is cache this pattern, but not that pattern ... cache GET, but not POST, etc. | 22:31 |
| JED3 | nkinkade: okay, lets doublecheck as to whether varnish uses q params in the cache's keys | 22:32 |
| JED3 | nkinkade: looks like get parameters are fine | 22:34 |
| nkinkade | JED3: How/where are you checking this? | 22:34 |
| JED3 | so if thats the case, only /rest/(dev|staging|1.5)/license/(standard|publicdomain|zero|recombo)/issue will need to be excluded | 22:36 |
| JED3 | nkinkade: looking at the varnish docs | 22:36 |
| JED3 | nkinkade: http://varnish-cache.org/docs/trunk/faq/general.html#how | 22:36 |
| JED3 | 3rd question, makes me believe that varnish keeps the query string by default | 22:37 |
| nkinkade | JED3: What goes to the API via those requests you mention that is user specific? | 22:39 |
| nkinkade | I thought all that stuff was handled via cc.engine. | 22:39 |
| nkinkade | Does cc.engine just make API calls? | 22:39 |
| JED3 | nkinkade: the engine doesn't use the api at all, except for the jurisdictions.js in the templates | 22:40 |
| nkinkade | So what requests are user-specific? | 22:40 |
| JED3 | nkinkade: as in which ones are highly parameterized? | 22:41 |
| JED3 | http://api.creativecommons.org/docs/readme_staging.html#license-class-issue | 22:41 |
| JED3 | http://api.creativecommons.org/docs/readme_staging.html#license-class-get | 22:41 |
| nkinkade | JED3: parameters don't worry me, just ones that return user-specific data. | 22:41 |
| JED3 | just those two | 22:42 |
| nkinkade | JED3: Then what is an example URI to the API that would break functionality if it were cached? | 22:42 |
| JED3 | well if /issue was cached then that would be an example | 22:43 |
| JED3 | that url is posted to with a set of user parameters | 22:44 |
| nkinkade | JED3: What might a full URL be that would break the system? | 22:45 |
| JED3 | so just issue actually needs to be exempt | 22:45 |
| JED3 | http://api.creativecommons.org/rest/dev/license/standard/issue | 22:45 |
| mralex | JED3: When the song picker list for Populist is long (as it is right now), it takes forever to display the popup and register a request - chrome timed out a few times | 22:46 |
| nkinkade | JED3: The above URL just returns "None" for me. | 22:47 |
| nkinkade | I guess I'm missing the params. | 22:47 |
| nkinkade | Oh, it's a POST. | 22:47 |
| nkinkade | I think I must be being dense here, because it's not clear to how the results of that URL will differ from one person to the next, as long as Varnish respects all query paramaters. | 22:48 |
| *** mralex has quit IRC | 22:50 | |
| *** JED3 has quit IRC | 22:50 | |
| CC_Hudson | Project cc.i18n build (43): SUCCESS in 28 sec: http://code.creativecommons.org/hudson/job/cc.i18n/43/ | 22:51 |
| CC_Hudson | Transifex robot: l10n: Updated Russian (ru) translation to 100% | 22:51 |
| *** mralex has joined #cc | 22:55 | |
| *** JED3 has joined #cc | 23:02 | |
| nkinkade | JED3: To be sure, as I'm writing the regex, for the URL portion, this would match all requests for /issue that should not be cached "/issue$" | 23:07 |
| nkinkade | That is, there will never be anything, even a slash, after /issue, right? | 23:08 |
| JED3 | nkinkade: nope that'd return 404 | 23:09 |
| nkinkade | JED3: In that case it seems as if we are safe to skip caching any requests to the API of the form /issue$ or /get$ | 23:10 |
| nkinkade | Does that sound about right to you? | 23:10 |
| JED3 | nkinkade: yeah, i actually think we can cache get, but better safe than sorry, dont cache issue or get | 23:11 |
| nkinkade | JED3: Out of ignorance on my part, how does /get differ from /issue, except that it take query params instead of POST variables? | 23:12 |
| JED3 | nkinkade: thats the only different, the request method | 23:12 |
| JED3 | difference* | 23:12 |
| JED3 | get is GET, issue is POST | 23:13 |
| JED3 | no redirection between the two either | 23:13 |
| nkinkade | JED3: So why would you think that the /get call could be cached, and /issue not? | 23:14 |
| nkinkade | Just because /get isn't really used by anyone? | 23:14 |
| JED3 | no because all of the parameters to /get are in the query string | 23:15 |
| JED3 | which is kept in the varnish cache key | 23:16 |
| JED3 | right? | 23:16 |
| nkinkade | JED3: That makes sense, you're right. | 23:40 |
| nkinkade | But I guess it doesn't really make sense to cache single use URLs anyway, so I just excluded /get? as well. | 23:40 |
| JED3 | nkinkade: awesome, is varnish enabled right now? | 23:41 |
| nkinkade | JED3: Yes. I re-enabled caching in Varnish. | 23:41 |
| nkinkade | And I verified it isn't caching for /issue and /get? | 23:42 |
| nkinkade | Seems to be caching for everything else. | 23:42 |
| nkinkade | Varnish is really great. | 23:42 |
| JED3 | nkinkade: great! | 23:42 |
| JED3 | yes it is. | 23:42 |
| nkinkade | I can't imagine using Squid for anything these days. | 23:42 |
| JED3 | nkinkade: why is that? | 23:43 |
| nkinkade | Because I don't know of anything that Squid does that Varnish doesn't improve upon. | 23:44 |
| nkinkade | Though I'm sure someone could find numerous things. | 23:44 |
| nkinkade | It's just for my/our use cases Varnish absolutely simplifies things. | 23:44 |
| nkinkade | The config is so easy. | 23:44 |
| JED3 | nkinkade: ahh yeah, the conf examples on the squid site look really complicated | 23:47 |
Generated by irclog2html.py 2.6 by Marius Gedminas - find it at mg.pov.lt!