Reasoning for putting Unleashed in a revisioning system like Subversion:
- It allows the developers to contribute directly.
- Direct contribution may make certain things easier, such as going through and properly localizing Puppy, and keeping it that way. (Obviously we'd have to watch the devs and yell at them when they don't localize their code).
- Removing the middle man removes opportunity for accidental errors.
- Removing the middle man lifts a lot of burden from the "coordinator", assuming we keep using coordinators. He doesn't have to manually add in every little change. If we stick with the coordinator approach, he would still make the decisions and stuff, just not have to implement everything himself. I'm sure WhoDo understands what I'm saying here
- Changes are recorded automatically. If you notice that something has broken in the latest release of Puppy, you can look up exactly what has changed in each file.
- By allowing anonymous reading, people can see what has changed on a day to day basis, and who changed it. They can test new features immediately, or simply satisfy their curiosity. If a bug fix is implemented, it is available immediately.
- Collisions are much less likely. A collision is when two people are working on the same file, and one submits their modified one after the other has already submitted his, causing the other one to be overwritten. It is difficult to keep track of that sort of thing when middle-men are involved. SVN will automatically notify you if the file has changed while you worked on it, and it will help you to merge the changes together, often taking care of it automatically (like if you were working on the bottom of the file, and the other person modified the top).
http://svnbook.red-bean.com/en/1.5/svn.basic.html
There are several options for what software to use. I'm leaning toward SVN, because that seems the most straightforward. Git has some fancy features, but we don't really need them AFAIK.
So, I've been working on setting up my old computer to serve as a test server while we figure out the best layout and stuff. Then, Caneri is considering hosting us on one of his new servers if they prove to be stable.
Issues/questions:
1: I was going to get it all set up tonight and then start this thread for discussion, but I ran into a snag: Neither SVN nor Git can handle device files (like in /dev/). We could get around that by keeping those in an archive, and having a script to unarchive them and remove the archive before building Puppy. We probably need a script to go from the svn layout to a proper unleashed layout anyways, so that could just be part of it. There could be another one to go in reverse. I'm going to play with this next time I have free time.
Does anybody have a better idea? Or know of another software that handles these files?
2: How best to handle upgrades to new base-versions of Puppy (such as after a rebuild from t2)? I think that the simplest method would be to single out any packages with significant Puppy changes (like 0rootfs_skeleton) and handle them by doing an svn copy on the old version to the new version's name, then replace the files with the new version. That way the histories for the files that are kept (like the wizards) would be preserved. Things with almost no Puppy changes might be easier to just import, I dunno. I still haven't gotten around to fiddling with T2, as I've been so busy with school. What are your thoughts?
3: Unleashed is big. We will need to set up a method to download the actual packages and such for unleashed from the package server like we do now, and then extract them and move them into the proper layout for SVN. Then do the svn checkout over those files, using the --force option. Then we'd have to do an 'svn revert' to bring it up to date with the server (when it checked out the files, it considered any differences in the local files to be new, not old, so we'd need to use revert rather than update). Finally, we may need to run a script to get a list of all files under revision control, and then go through the tree and remove any that aren't. Otherwise if a file that existed when the package was made gets removed later, it will still hang around because SVN will ignore it. This doesn't cause any problems directly, but it could be confusing to the user, as he may not realize that the file isn't being used, or may think that the file belongs and re-add it, causing issues. All this stuff can be scripted of course, so that the user just has to run the script and it will take care of everything.
Maybe doing all that isn't necessary, but I think it might be more efficient than just checking out the entire system. I think a checkout would download a lot more small files, rather than the smaller number of package files. It depends on how SVN transfers its data I suppose. Anybody know which would be more efficient?
4: What file layout would be best? I suspect that it would be easiest to have the standard 'tags', 'branches', and 'trunk' directories at the top, then put the main line in 'trunk'. This way when doing a major upgrade of Puppy, it can be done in /branches, and when a release version is ready, the whole thing can be put into /tags with one command. This makes it seem like maybe branching a single package wouldn't work well, but SVN is very flexible. You can simply branch it to /branches/my_branched_package. SVN doesn't actually care if the layout of everything in /branches matches /trunk. When you finish, just remove it from /branches. Tagging packages would be more of an issue, but wouldn't really be necessary afaik. If the package reaches a good stable stopping point before the next version of Puppy is ready, just copy the package over to a new version number and keep going, leaving the packages.txt file pointed at the old stable version until the new one becomes stable. Every now and then we could prune out the old unused versions, just to make things cleaner. The wouldn't actually be removed though, since it's SVN, so if you need the old version you can just go back and get it.
I also think that the packages directory should be split out from the unleashed_core. For people who maintain a full local tree, it makes no real difference. But for people who just want to check out the unleashed_core so they can work on it don't want to have to do fancy commands to skip checking out all of the packages. So separating it into a packages directory containing all the packages and an unleashed_core directory for the core would correct this. They can be put back together easily when it's time for a build, or simply simlinked.
So the overall structure I'm envisioning would be like this:
Code: Select all
bash-3.00# tree
.
|-- branches
|-- tags
`-- trunk
|-- packages
| |-- 0rootfs_skeleton-4.02
| |-- pkitchensink-9.54
| `-- some_prog-0.24
`-- unleashed_core
|-- boot
|-- isolinux-builds
|-- kernel
`-- packages -> ../packages
Any other thoughts? Problems? Better ideas?
5: How will something this large handle? That question is the main reason I'm setting up my test server. I want to experiment with it to see what kind of performance I get. I also want to simulate a large number of changes and see how fast the size grows. It's running on a 450MHz PIII, so it isn't the fastest machine in the world, but it's reasonable. If it performs well on this, it ought to be just find on Caneri's server, CPU wise.
Initial numbers: it looks like when you take a 996 MB unleashed tree and import it into SVN (minus the device files, which are probably insignificant), it only seems to use 269 MB. So it looks like it compresses stuff. That's good. I was expecting it to be 10% more than the original, but it's actually 75% less than the original.
6: Is using the svnserve program good enough (for now)? Does anybody think we really need to be using encryption to contribute? Does anybody think that they in particular will need an encrypted connection (oppressive laws or whatnot)? I do plan to work out a web frontend, but that's low on my priority list.
I have the test server running and most of the data loaded in. I need to get a little more done before I can make it public though, and I won't have time tonight. I have to go decide what classes to take next semester, which is harder than it sounds - have to make sure the schedules work out properly. Pain in the butt. Then if I don't get in and sign up immediately, a key class may be filled up and have broken the entire schedule. I think I'm in the very first group to sign up this year though, so hopefully that won't be an issue.
How many people are interested in playing with the test server? If you tell me now I can get you all usernames right away. Or should I just make a generic test user and post the password here? It is only a test server, so that's an option for now. Unless people start abusing my bandwidth. I'm on my campus's network, and the policy here is that hitting a GB a day more often than once a week or so is abusive.
Hopefully I'll be able to finish up on Tuesday. Nearly there now, but I'll probably spend all evening tomorrow doing homework.
So, discuss! Any deal-breaking problems I've overlooked? Any better options? Any suggestions? Speak boy, speak.
Good doggy
Oh, and to head off the "SVN has been tried before, didn't work" argument: No, it has not. Not like this, anyways. An SVN for Puppy-related programs is one thing. An SVN for the official Unleashed tree is quite another. Until now, Barry has done most of the work, and he did it himself. Now the Community is supposed to do the work. What better way than to simply allow the Community to do the work? As opposed to picking one person to shoulder the burden and try to put together everything that everybody is submitting to him? No. Let the devs develop, I say.