Monday, 2009-02-23

*** jed3 has joined #cc01:27
*** mlinksva has quit IRC04:33
*** everton137 has quit IRC04:46
*** Roderick_ has quit IRC05:21
*** sama has joined #cc06:18
*** parkerhiggins has joined #cc06:27
*** parkerhiggins_ has quit IRC06:44
*** nkinkade has quit IRC06:51
*** paulproteus has quit IRC07:05
*** paulproteus has joined #cc07:06
*** funky_cold has joined #cc08:41
*** balor has joined #cc09:39
*** sama has quit IRC09:46
*** Danny_B has joined #cc10:24
*** sama has joined #cc10:38
*** tvol has joined #CC13:58
*** jed3 has quit IRC14:22
*** johndoigiii has joined #cc14:23
*** mlinksva has joined #cc15:09
*** nathany has joined #cc15:24
*** luisv has joined #cc15:26
*** nkinkade has joined #cc15:40
*** stevel has joined #cc16:15
nathanymorning, nkinkade16:21
nathanywhat's your day look like?16:24
nkinkadenathany:  good morning.16:31
nkinkadeThe early part answering email, then I was going to think about the wikis.16:31
nkinkadeWhich I wanted to talk to you about.16:32
nathanynkinkade: right, just saw your email16:38
*** luisv has quit IRC16:44
*** rohitj has quit IRC16:51
nathanynkinkade: i just replied; let me know if it doesn't make sense16:52
nathanymorning, johndoigiii -- welcome :)16:52
johndoigiiigood morning to you as well16:53
nkinkadeHi johndoigiii.16:53
johndoigiiihello nkinkade16:53
nkinkadejohndoigiii: Are you in SF now?16:54
johndoigiiiNo, still in Raleigh NC16:54
nathanyhe'll be in the office next week16:54
paulproteusHowdy doody, all.16:59
*** jgay has joined #cc16:59
nkinkadeHow do, paulproteus.17:00
paulproteusWelcome johndoigiii (I'm the Asheesh who chatted with you on the phone).17:01
johndoigiiiHello Asheesh17:01
*** nathany_ has joined #cc17:02
*** nathany has quit IRC17:03
*** nathany_ is now known as nathany17:03
johndoigiiiso does the submissions work?  nathany described it as "bonked" and I am having difficulties submitting an issue17:04
johndoigiiii could very easily be formatting these message incorrectly though17:05
nathanyjohndoigiii: still borked17:05
nathanythe problem is that we require a project and priority to create issues17:05
*** stevel_ has joined #cc17:05
johndoigiiinathany: ahh yes, thats what my return messages are stating17:05
nathanyand don't have a sane default for project, i guess since that's the field we added to their schema17:05
nathanyi'm trying to remember if the default schema has a default for priority17:05
nathanyactually i'll take a quick look at that now that you mention it17:06
*** TRD has joined #cc17:08
*** everton137 has joined #cc17:12
*** stevel has quit IRC17:18
*** sama has quit IRC17:19
*** stevel_ has quit IRC17:20
*** stevel has joined #cc17:20
nathanyjohndoigiii: it should work now17:20
nathanyit'll create an issue w/o a project but won't let you edit it through the web interface w/o adding one.17:20
johndoigiiinathany: oh okay, I had one go through just a moment ago, but I specified a proj17:21
nathanyyeah, you can definitely do that17:21
nathany(and it's probably a little better, fwiw, but it's nice to be able to cc the tracker on "normal" email)17:21
johndoigiiicool, I'll go ahead and delete my issue17:23
*** Bovinity has joined #cc17:35
*** robmyers has joined #cc17:58
*** sama has joined #cc18:35
*** K`Tetch_ has joined #cc18:55
*** everton137 has quit IRC19:14
*** rohitj has joined #cc19:33
nkinkadenathany: Any reason I shouldn't add languages to the ccnetwork project in Poolte?19:54
greg-gew, pool tea doesn't sound good at all19:57
Bovinityonly from the freshest pool water20:03
nathanynkinkade: non20:18
nathanyno, that20:19
nkinkadeI figured, so I already added the language.20:19
nkinkadepaulproteus:  Your modified git-mergetool, I forgot how it worked.  Something about "theirs," right?20:39
nkinkadeNevermind, I found the email.  Forgot you had emailed me directions.20:42
nkinkadepaulproteus:  nathany: In trying to merge MW REL1_14_0 in the cc_skeleton branch of mediawiki.git, I get 277 conflicts.20:54
nkinkadeThis is not unlike the experience with CiviCRM.20:55
nkinkadeA lot of the conflicts are in the Messages<lang code>.php files, but it's hard for me to know which changes we need to keep.20:56
nkinkadeAnd it's only merging from 1_13_2 to 1_14_0.  This process of using git seems like a hassle in many ways.20:57
nathanyI can just about guarantee they're all spurious -- those are the equivalent of PO files, IIRC20:57
nathanypaulproteus: thoughts?20:57
nkinkadeRight, but there are other legitimate files and I don't know which changes were done by us.20:57
nkinkadeI could find out by looking through history, but that seems crazy and laborious.20:57
nkinkadeI wouldn't have been able to do the git merge for CiviCRM without the intervention of paulproteus.20:58
nkinkadeAnd him making some changes to git-mergetool.20:58
nathanymy understanding was that using git-svn meant git was aware of the changes between, say, 1.13.2 and 1.14.0, so that it would be able to merge sanely; paulproteus?20:59
nkinkadeThat was my thought as well, but this Mediawiki merge and the CiviCRM merge were packed with conflicts.20:59
nathanylet's wait for him to get back and see if he has insight into why that is21:00
nathanymaybe we should be rebasing instead of merging21:00
johndoigiiihey everyone, I having been trying to get commoner to build out, but keep receiving errors when trying to fetch the lxml dependency.21:00
nkinkadenathany:  I'm just going to do the monitor install from the git repository, but rather than any merging, I'll just do a checkout of 1.14.021:01
johndoigiiianyone have experience installing this library on Mac OSX?21:01
nkinkadenathany:  That will mean that we'll miss out on any mods to the code that we have in cc_skeleton, but I don't know how many we have anyway.21:01
nkinkadeWe could also do this with subversion, and just track their tree directly and at upgrade time do an svn switch.21:02
nkinkadeWhy are we doing it with git-svn, again?21:02
nkinkadeHonestly, I can't remember.21:02
nkinkadeI've learning to detest Mediawiki as it is, but things like this make me dislike it even more.21:03
nathanynkinkade: we're doing it because it lets us do patching/mods as needed which svn:externals doesn't21:03
nathanyjohndoigiii: what error are you getting?21:04
nathanyi have built it on OS X, but i think my setup is a little non-standard21:04
paulproteusnathany, rehi21:04
paulproteusSo that we can have local commits, for example.21:04
paulproteusAlso because I'm a git fan.21:04
paulproteusI can give other reasons I'm a git fan, like it's fast, and I Really Understand it.21:04
paulproteusFor example, we have that source patch in the ccLearn wiki.21:04
paulproteusExcept that one's not gitified, so it just sits there as an uncommitted change in an svn repo.21:04
paulproteusnkinkade, iirc upgrading between point releases was very easy.21:04
nkinkadepaulproteus: Then maybe upgrades of Mediawiki are something that you can handle.21:05
paulproteusThe history with git-svn plus branch switches by upstream gets messy because of differences in the git and svn history model, I suppose.21:06
nkinkadeI Don't Really Understand git, nor do I have a strong desire to.21:06
nkinkadeI only want to know enough to use it for basic things, and beyond that I'm donewith.git21:07
nathanyjohndoigiii: i'm looking at it now21:07
nathanyi suspect the problem is either a) missing headers or b) version issues21:07
nathanyare you running using the Python interpreter Apple ships?21:08
paulproteusWhoa, a Mac user? Freaky-deaky.21:08
paulproteusnkinkade, I don't want you to live a life of aggravation any more than you have to.21:08
nkinkadepaulproteus: So you think the conflicts are the result of something being lost in translation with git-svn?21:09
paulproteusIt's the opposite; it's that git-svn uses information svn ignores....21:09
paulproteusBut 'rebase' might be a better plan than 'merge'.21:09
paulproteusLet me give that a shot to see how it goes.21:09
nathanyjohndoigiii: so it's *at least* a version issue; according to the pastebin (thanks for that, btw) you have lxml 1.1.12 and lxml needs 1.1.15 (
paulproteusnkinkade, But on the other hand, if you understood git, you would gain more effective access to its power.21:12
*** rohitj has quit IRC21:12
nathanyjohndoigiii: i tend to build a non-stock Python on OS X and install libraries like libxml2 and libxslt1 into my home directory (--prefix=~)21:13
nathanyi'm on a call right now, but maybe creating a new user to test that with would be one way forward?21:14
johndoigiiialright I will give that a try21:14
nathany(if you don't want to pollute your current home)21:14
nathanyi'll ping you when i'm done with the call to see where you are with it21:14
johndoigiiisounds good, thx21:14
*** TRD has quit IRC21:15
nkinkadepaulproteus: Did rebase help?21:34
paulproteusnkinkade, I'm going to have to finally do what I hoped I wouldn't have to do, which is devise a sound, sane workflow for git-svn based upgrades.21:34
paulproteusI'm experimenting with that now.21:34
paulproteusI believe I understand the problem well, anyway.21:34
nkinkadeCool.  Thanks.21:35
nkinkadeI've got 6 MW installs in the queue, which makes me somewhat sick in itself, but the thought of having to upgrade a dozen or so MW installs every so often is even more sickening.21:35
nkinkadeActually, installing is easy, it's the upgrading that is just time-consuming.21:36
paulproteusI'm doing:21:39
paulproteus$ git diff origin/tags/REL1_13_2..cc_skeleton | less21:39
paulproteusjust to get a sense of what the patch entails.21:39
paulproteusOops, piping to less is a mistake.21:40
nathanypaulproteus: does the .. syntax mean between 1.13.2 and cc_skeleton?21:41
nathanyi haven't seen that before21:41
paulproteusLooks like I can drop the ".." and just put a space there.21:41
paulproteusThat's more sane and I will always do that from now on.21:41
paulproteusSo ignore the new syntax you learned and pass in separate arguments, I'd say.21:42
paulproteusnkinkade, Because we have the local commits, here's what I imagine would be a reasonable UI.21:43
paulproteusFirst you get presented each patch we have against the current release and the commit log message explaining it.21:44
paulproteusYou get asked, "Do you want to ditch this?"21:44
paulproteusSince we here in reality, where upstream takes some things but not others, the more patches you ditch, the easier the merge will be.21:44
paulproteusAnd then it tried to apply each to the new release, and where there's a conflict, it lets you edit.21:45
paulproteusYou do this once per release, and then you can just "pull" from each of the MW installations we have.21:46
Bovinityis it our office connection, or is still heinously slow?21:46
paulproteusDue to reality, I don't think that anything much less complex has a hope of working.21:46
paulproteusSeems decently fast to me from home, Bovinity.21:46
nathanyBovinity: seems semi-snappy21:47
nathanyis it because you're logged in?21:47
Bovinityyeah, it goes a tad faster when i log out.21:50
nkinkadepaulproteus: What changes have we ourselves made to the MW code?21:51
paulproteusSo there's the obviously-ours stuff, which is files we added that aren't in upstream (like LocalSettings.php).21:52
paulproteusBut we've modified EditPage.php and GlobalFunctions.php with trivial whitespace modifications.21:52
* paulproteus keeps reading21:52
paulproteusWe seem to have added Image.php out of nowhere, presumably because it existed in an older revision of MW and we didn't lose it at an upgrade.21:53
paulproteusWe've appended to the end of Setup.php as well as the middle of it (I have no idea why).21:53
paulproteusincludes/SkinTemplate.php we have some odd changes to.21:53
paulproteusWe seem to have created includes/SkinTemplate.php.copy which may contain the original SkinTemplate.php.21:54
paulproteusYou can see this yourself with the "git diff" command I gave above, so long as you have a local branch cc_skeleton that points at the same rev as cc_skeleton in the repo.21:56
nkinkadepaulproteus: I do have that local branch.21:56
nkinkadepaulproteus: Earlier, before you posted your comments about git diff, I had got some diff up by doing "git checkout cc_skeleton; git diff origin/tags/REL1_13_2" and it seemed to have useful diffs.21:57
nkinkadeBut it may not have been the diffs I thought I was looking at.21:57
paulproteusThat is what you want, though possibly reversed.21:58
paulproteusSame thing, except possibly reversed (and it's easy enough to tell from the diff output).21:58
nkinkadeIt seemed normal, not reversed, that is.21:58
nkinkadeThere were pluses by the things I new we had added.21:58
* paulproteus nods21:59
paulproteusWe change e.g. maintenance/
nkinkadepaulproteus: So there's really no way around the conflicts and the resulting need to go through >250 of them manually, at least without special scripting each time?22:02
paulproteusThat's not what I'm saying.22:02
paulproteusThe language files, for example, are due to the mismatch between what git should be doing and what it is.22:02
nkinkadeSo you're saying that we blindly accept change to files we know we haven't touched, and then manually look at ones that changed that we also modified?22:03
paulproteusI suggested a "reasonable UI" about half a page up.22:03
paulproteusWould that work for you?22:03
nkinkadeI think that would be fine.22:04
paulproteusCool. I'll go back to thinking about that for a bit.22:04
nkinkadeOne potential problem is when some change implemented upstream could conflict with what we've changed.22:04
paulproteusThat's unavoidable, yeah.22:04
paulproteusBut I *firmly agree* with the need to not pain the user with spurious conflicts.22:05
nkinkadeAnd since I have no idea whatsoever why most of the changes we made were done, then I'd have no way to how to handle things.22:05
*** rohitj has joined #cc22:05
nkinkade(to *know* how)22:05
nathanynkinkade: sure, but that's sort of a completely different issue :)22:06
paulproteusJust throw away the changes you can't justify, and email webmaster saying "this is what I did; suck it".22:06
nathanypaulproteus: does doing a diff between the previously imported version (say, 1.12) and the newest version (1.14) give us anything useful?22:06
nkinkadeIt may be 2 issues, but they are intertwined.22:06
nkinkadeProbably just about 300 more conflicts.22:07
paulproteusnathany, Well, here's the thing. git doesn't intrinsically see the difference between MW-official changes and CC additions.22:07
nathanyi'm just thinking about a diff that might apply more cleanly22:07
nathany(for some definition of cleanly)22:07
paulproteusRight - you're saying, apply the diff between LAST_MERGED and LATEST_RELEASE on top of the skeleton?22:08
paulproteusTrying now.22:08
*** sama has quit IRC22:08
nathanymy only thought is that it'd be good to avoid building lots of custom tooling if we can possibly avoid it (where lots is a value > 0)22:09
nathanyjohndoigiii: any luck with the lxml stuff?22:10
*** stevel_ has joined #cc22:10
johndoigiiiI feel like I am getting closer, I've installed later versions of the libraries22:12
nathanyis it still barfing when it tries to build the egg?22:12
johndoigiiistill seeing the older versions22:12
johndoigiiitrying to set up aliases22:13
johndoigiiifollowing these instructions
*** stevel_ has quit IRC22:13
*** [mharrison] has quit IRC22:13
*** stevel_ has joined #cc22:13
paulproteusnathany, Conflicts still with your approach.22:13
johndoigiiithis isn't working for me, so I just added a new user and am going to try your idea22:13
paulproteusGoing to not look at IRC or email for a bit and focus on this; ping me if needed.22:14
*** stevel has quit IRC22:14
nathanyjohndoigiii: yeah, that looks like it'll work but sort of gets you involved in Framework -hell (the distant cousin of dll-hell)22:14
johndoigiiiyeah tell me about it22:15
nathanyone other thing i've found useful is setting LD_LIBRARY_PATH before i run the buildout22:15
johndoigiiiyeah I read that as well22:15
nathanyI can't remember if that works on OS X or not... but i seem to remember it helps make it find the libraries i want first22:15
paulproteusnkinkade, Okay, so if you totally abandon our changes in that one massive "Added CC-specific changes to the base MW install" commit, moving our changes forward is easy.22:24
paulproteusFor another thing, I think it makes way more sense to use the concept of rebase rather than the concept of merge, even though the vocabulary isn't as clear.22:25
paulproteusI can provide to you a sensible, comprehensible explanation for all of this if you're willing to stay with me and ask questions when it seems to make no sense.22:25
paulproteusAnd it requires zero custom tooling, and can be achieved mostly by right-clicking in gitk and clicking "cherry-pick this commit".22:25
paulproteusNote that some of the clicking could be automated away, but it's not so bad really.22:26
paulproteusI also think no one will miss those old changes, and if they do, then by Jove they can make commits with good commit log messages to explain why they wanted them.22:26
nkinkadepaulproteus: That seems fine.22:26
nkinkadeSo, I could use your modified git-mergetool to just always accept their changes: "theirs"22:27
nathanypaulproteus: can you send a message to webmaster with a link to that commit diff on viewgit?22:27
paulproteusnathany, Heck yes.22:27
nathanythanks -- that way people can skim it22:27
paulproteusnkinkade, No, you're going to have to move to a different workflow that doesn't involve the word merge at *all*.22:27
nkinkadepaulproteus:  Ah, rebase.22:27
paulproteusUnfortunately, this will require creating a new name for the CC skeleton every time we switch what we're based on.22:28
paulproteusOr, well, not doing that, but being careful instead.22:28
paulproteusYou know, I need a haircut.22:28
Bovinitybash 4.0!22:28
nkinkadeIs rebase not dissimilar to svn switch?22:28
paulproteusrebase is totally similar to svn switch.22:29
paulproteusnkinkade, Here, let me have you try something.22:29
nkinkadenathany: So, until we get some of these details hammered out, how do you suggest I proceed, since we need some of these wikis sooner than later?22:30
nkinkadepaulproteus: Shoot.22:30
paulproteusI can hammer these details out right now with you.22:30
paulproteusnkinkade, $ git branch new-skeleton origin/tags/REL1_14_022:30
paulproteus$ git checkout new-skeleton22:30
paulproteusNow look in gitk --all, and look for the cc_skeleton branch pointer.22:30
paulproteusNote that every point you see on the left is a commit. Most of the recent history of cc_skeleton is merge commits.22:31
paulproteusBefore the merge commits, back in July 2008, you'll see some actual commits, including the infamous "Added CC-specific changes to the base MW install".22:32
paulproteusThe best way to go backwards is to click in the Parent: XXX links.22:32
paulproteusYou can use the forwards and backwards buttons as if gitk were a web browser.22:32
paulproteusLet me know if you see or fail to see those.22:33
nkinkadepaulproteus: I see those things.22:41
nkinkadeSo now I just cherry-pick the commits I had made?22:41
paulproteusYou can right-click them in gitk and hit cherry-pick, even!22:42
paulproteusOr you can "git cherry-pick REVISION".22:42
nkinkadeSo basically I'll need to cherry-pick each commit that was made by a CCer?22:42
paulproteusYeah, but now you'll have beautiful linear history rather than history that snakes into the past, tied down with merge commits.22:43
paulproteusOh, and test. (-;22:43
paulproteusI mis-spoke earlier: "git checkout" is like "svn switch", really.22:44
nkinkadepaulproteus: Why are the commits not chronologically ordered in gitk near the time of our changes?22:45
paulproteusThey're in commit history order, constrained by the merges.22:47
paulproteusI hope that makes *some* sense?22:47
paulproteusFor example, if I do a merge of REL1_13_1 a year after 1_13_1 happened, that shows up right on top of 1_13_1.22:48
paulproteusBut also so does the MediaWiki commits that took place right after 1_13_1.22:48
paulproteusSo that would cause chronological mis-ordering.22:48
nkinkadeAh.  That makes sense.22:51
paulproteusHooray, sense!22:51
paulproteusThis is git-svn MW workflow version 2.22:52
nkinkadepaulproteus: When I tried to cherry-pick the "Added CC-specific changes ..." I get an error popping up.  Lots of conflicts, then a child process exited abnormall."22:52
paulproteusThat one's a total nightmare.22:52
paulproteusGrab what you need from it, and ditch the rest.22:52
paulproteusLike LocalSettings.php.22:52
nkinkadeThis seems error prone.22:52
*** johndoigiii has quit IRC22:52
paulproteusDitching that horrible commit is a *goal* of redoing the workflow.22:52
*** johndoigiii has joined #cc22:52
paulproteusNo more commits in the CC MW history that can't be explained.22:52
paulproteusLet's end the insanity once and for all, or else it will haunt us forever.22:53
paulproteusI'd say "git cherry-pick" it on the command line, and as it fails, undo the parts you don't want.22:53
paulproteusI'd offer to do that for you, but you should probably be familiar enough with this stuff to do that.22:54
paulproteusI'll do it with you so I can quickly answer questions, though.22:54
paulproteus(Does that also make sense as a plan?)22:54
nkinkadeHow do you plan to go through it with me?22:55
nkinkadeAnd you're saying I should cherry-pick that commit, let it blow up and then manually reset the files we don't want?22:56
paulproteusnkinkade, Yes re: "And you're..."22:56
paulproteusgo through it with you: via IRC, I was thinking22:56
*** robmyers has quit IRC22:57
paulproteusBut I'd do the same commands as you...23:00 know, why don't we screen-share?23:00
paulproteusThen you can just say "wtf" every few minutes on IRC and I'll see what's up.23:00
*** johndoigiii has quit IRC23:00
paulproteusHow about I join you in a "screen -x" on one of the CC servers?23:00
*** johndoigiii has joined #cc23:00
nkinkadepaulproteus: We could do that.  I'll try it here on my local copy first.23:01
paulproteusOkay, cool.23:01
nkinkadeAt the moment I'm just manually resolving the conflicts from the merge of that commit.23:01
paulproteusFrom the cherry-pick.23:02
paulproteusAnd hopefully you're erasing almost all of it.23:02
paulproteusSince that commit is the spectre that haunts our lives.23:02
nkinkadeI'm dumping things from that commit and keeping what's in HEAD, unless it's obvious it's a change we made.23:05
paulproteusSuper rad.23:06
nkinkadeBovinity: Do you know of changes that have been made to the code for wiki that need to remain, not including the CC4 skin?23:21
nkinkadepaulproteus: nathany: I'm resolving these conflicts manually and most of them seem like things we never altered ourselves.23:22
nkinkadeDo either of you know of any secific changes we've made to the MW codebase?23:22
nkinkadeI'm guess what I'm wondering here is if we're better of just starting with a totally clean 1.14 checkout and go from there.23:23
nkinkadeAnd then do commits as we need them, and in a more incremental fashion.23:24
paulproteusnkinkade, I agree, and that's my point; that's what you're doing if you cherry-pick only the changes we want.23:24
paulproteusThe "changes we want" mostly are the submodules.23:24
nkinkadepaulproteus: I'm saying not to cherry pick anything at all, including the submodule stuff.23:24
nkinkadeBecause I'd like to redo the submodule stuff anyway.23:24
paulproteusOkay, fine with me.23:24
paulproteusAnd that will be your way of getting us clean history, and throwing away all CC stuff that doesn't have clean commit history.23:25
nkinkadeI'm just going to do it that way.  We've got the git history if it turns out we needed some of those changes, but I see that as unlikely.23:25
nkinkadeIt's likely that those "CC-specific" changes were CC-specific at all and were just cruft left over from a years of upgrading.23:26
*** stevel has joined #cc23:30
*** mlinksva has quit IRC23:43
*** stevel_ has quit IRC23:46
paulproteusnkinkade, Lemme know once you push, and I'll make sure we say something sensible to webmaster.23:49
paulproteusAs we embark down the road toward sanity.23:49
nkinkadepaulproteus: I was thinking of doing the same type of setup like we have for CiviCRM, with cc_production and cc_staging branches.23:50
nkinkadeDoes that seem right to you?23:50
nkinkadeAlso, for the submodules, I'm only going to create submodules for those extensions that actually have their code in some type of repository.23:51
nathanypaulproteus: libapache2-mod-fastcgi is the chinese variant in ubuntu/debian?23:51
paulproteusNo! That's the evil American one.23:51
paulproteusNote that the American one is non-free.23:52
paulproteusThink about *that*....23:52
* paulproteus giggles.23:52
nkinkadeIf the extension is just an unversioned PHP file or code that you cut-n-paste from the MW site.23:52
nathanypaulproteus: btw, i keep seeing on this ubuntu box i admin23:53
nathanyany ideas?23:53
paulproteusdpkg-reconfigure locales # and check the box for that locale23:53
nathanypaulproteus: ^^23:54
nathany(there was no box to check)23:54
nathanypaulproteus: manually installing language-pack-en seems to have fixed it23:59
nathanythanks :)23:59

Generated by 2.6 by Marius Gedminas - find it at!