37

i went to the gllug meeting tonight about filmlight’s baselight digital film grading system. it was really interesting. the presenter gave a brief overview of film and digital workflows too. you can read more about the system and architecture on the website, but the basics are that it uses a linux cluster with each node having a graphics card for output through dvi. they then use a custom built piece of hardware to combine the individual outputs to a full frame. each node works on a piece of the frame (1/n depending on how many nodes are used: 2, 4 or eight (i can’t put the number because wordpress automatically turned it into a smilie)). it seems that the movie industry is one place where there are a lot of interesting projects. he even mentioned that they’re looking using a new tyan motherboard that takes 8 dual core opterons.

i would say that a big plus to living in london is that there seems to be a good number of movie related companies without having to live in hollywood as well as enough linux/tech people to have good turn out for meetings. there was probably about 20 or so people at this meeting and there was a javascript meeting in the same building that had around 150 sign up. i had signed up for the javascript one at first, but didn’t realize it was the same night as this one.

the meeting was at westminster university’s new cavendish street campus again. i think the goodge street tube stop area would be a good one to live near. i think i would probably choose it over angel if i was looking right now.

update: i forgot to mention an interesting point from the guy from filmlight. one of the few proprietary bits of software they had to use was the nvidia video card driver. he actually said that they would trade performance for a card with free software drivers since it would allow them to fix problems and see the entire path. he also complained about binary drivers for talking to proprietary sans for data storage. they were apparently even worse by only supporting certain kernel versions, etc. i just don’t understand why any hardware company would want to make it more difficult to use/buy their hardware?

before even getting into this, it might be a good idea to read these two essays by paul graham: “revenge of the nerds” and the follow up “re: revenge of the nerds“. the specific section may be “what made lisp different” from the first, but you should really read the whole thing. also, the book i read is “practical common lisp“.

why not java?

it’s hard to imagine that i first started playing around with java in probably 1995 when it was still in 1.0 betas. so basically, a decade. in that time, java has definitely became popular, but simultaneously become worse. from my perspective, it’s fairly easy to point out what the problems are. for starters, the java language is completely tied to java the platform (all the api’s). the downside to this is that a large set of these api’s were written by sun at a time when sun wasn’t writing production programs (have they ever?) with it and the ones they did write (hotjava, a couple of failed development tools) completely sucked and failed. the first step to fixing this would be to create a core java language and api set similar to what is used for the j2me or personal java (old version). nothing else should be included in the standard distribution of the language. all the other pure java libraries should be distributed separately. i guess everyone could just switch to j2me as the “real” java core. however, used add-ons like jdbc aren’t packaged separately. i could be wrong on this and have not extensively looked at j2me lately so i’m not sure what the exact status is of the situation. there may be one of the profiles for j2me that is reasonable. it doesn’t negate later points.

a consequence of the development path of java is that there are multiple different ways of doing the same thing in java. a.w.t vs. swing, java.security.* (certificates, acls, etc). the list goes on. none of this cruft will ever be removed from the platform in the name of backwards compatibility (of course, that’s complete bullshit because they have changed the way functions behave (getTimeZone for example) which is a greater sin than removing deprecated, unused, duplicated api’s because at least removing something gives a compiler warning or runtime exceptions.). now there’s this whole process for adding new api’s to the standard platform that seems to amount to design by committee by competing vendors of existing software. sure, that’s a good idea. let’s get all the database vendors into the same room and have them create a common api for database access. whatever. not to mention the inclusion of things like the corba orb that i don’t think i’ve ever heard of anyone actually using. rmi should equally be scrapped. it’s interesting that sun completely ignored their own research and included rmi from the start and later added corba.

another issue for java is that it’s not freedom software. the constant stream of crap coming out of sun is that their customers like that it’s controlled and reliable and don’t want it to fork. i suppose they would like you to just ignore perl, python, ruby, etc. in their logic. the reason they’re scared of someone forking java has nothing to do with microsoft. microsoft has already forked java and called it c# (i’m not going to get into the fact that microsoft completely blew an opportunity with a crap fork). the real reason is that sun does such a shitty job of managing the situation that someone sure as hell will fork it and then everyone will abandon sun’s bloated crap java for something better. for starters, they would scrap a.w.t and swing and use s.w.t. for guis.

i don’t really want to go down this path. i could complain forever. those 2, slightly related, issues are really enough for me to jump ship. i agree with paul graham’s assertion in his essay “the hundred-year language” that java is an evolutionary dead end. i would much prefer to go down a more positive path of what is desirable to me in a language.

what do i want?

1. no fucking hump case/camel case/mixed case bullshit
2. i want to put ‘-’ in variable and function names
3. no checked exceptions
4. interpretable
5. i don’t want separate “languages”/formats for config files (properties files, xml), data (xml, csv, key/value, random), code (java, javascript, whatever), etc and associated parsers.
6. one parser for everything
7. more power
8. interactive
9. no xml anywhere near the core
10. no bloated core/ simple base
11. class definition optional
12. unspecified, unrestricted number sizes
13. strong typing, but not static typing
14. defined system for loading libraries
15. method equivalent to “java -jar foo.jar” to start a program with all resources (images, etc) included
16. ability to remove grunt work and replace incompetent plug-and-chug programmers with code generators.

so, that’s the basic list. we’ll assume that i don’t have to go through and point out where java fails on these points. this listed was made mid-way through “practical common lisp”, but these were things i had already previously desired. hopefully, i hadn’t been subconciously influcuenced by my lisp reading.

how does common lisp stack up?

1. all variable and function names are symbols which are typical written in all lower case (+) and internalized to all uppercase (don’t care). they are therefore case insensitive. you could apply the similar arguments for case sensitivity as you would to filenames. i would prefer only lowercase.
2. pretty much any character appears to be legal. ‘-’ is the normal word separator for all symbols (”string-trim”, “string-not-equal”). additionally the function name for addition is “+” and likewise for other math type things. using “->” for conversion functions also appears to be common, “object->string” to convert an, shockingly, object to a string. that’s not an actual existing function by the way.
3. there’s actually not really “exceptions” as normally viewed at all. instead, there is a much more flexible condition system. this allows code to signal conditions that can then be handled in some way by the calling code at any level. the most interesting feature of this compared to any other programming language that i’ve seen, is that the signalling code can provide “restarts” that the caller can use to allow the code to continue on from its point of exception in different ways. very cool. there’s been many times i’ve wanted to do this in java, but it’s just not possible. you then have to split up statements into multiple try/catch blocks which is really annoying and makes the code less readable.
4. while most implementations appear to compile to native code (good speed), there’s nothing stopping you from directly loading a source file and executing it directly. this is really the key point. not really to be interpretable, but to be able to load a file directly with out an intermediate step of compilation. how that is handled, i really don’t care as long as it occurs in a reasonable amount of time.
5. this is somewhat an extension of the interpretable issue. i don’t what to learn multiple speaking languages so why would i want to learn multiple programming lanuages, data formats, etc. the next person that tells me xml is the greatest thing in the world and does things that no other format does will probably get punched in the face. likewise, for anyone who wants to claim that xml is great for a computer to computer format. it’s funny (in the “that’s terrible” way) that people will claim that xml is so great because it’s human readable (because it’s text based) and also easily computer readable (because it has a spec). neither is really a good argument. for one thing, xml is not humanly readable for anthing greater than a couple of tags. it’s also not any good for computers because writing a parser is non-trivial, slow and the format contains a lot of redundancy. this isn’t about xml so i’m not going to go on, though i could. everything in lisp is s-expressions. code is data, data is code. everything is the same format. the one issue regarding this may be with limiting access to only certain objects (say for config files or a script loaded off the internet, sandboxing), but i haven’t explored this further. there’s always a way.
6. related to languages formats. could be considered the same point. like i said, everything is s-expressions.
7. i’m not really sure what “more power” meant. i had had a couple of pints when i started this a while back. we’ll just say that the ability to extend the core language with regular user code and have it seamlessly intergrate gives “more power”. you don’t have to wait on sun or anyone else to add a language feature/construct that you need. lisp is also good for creating mini-languages which is all api/libraries really are. it’s just that they don’t intergrate into the language well.
8. the ability to start up an interactive session is very nice. this is provided by probably ever scripting language. you type in a command and it executes, like a shell. it makes it easy to experiment. you don’t have to create a new file, a bunch of class stuff. you just type it straight in and your done. good for learning languages as well.
9. i guess this really goes back to the data format issue previously discussed where i got of track. i think the bullet is pretty clear and lisp doesn’t contain an xml parser in the core language.
10. the common lisp standard is supposedly ~1000 pages. seems a bit much, but the arguments i’ve seen claim that it includes a lot of examples and base functionality that smaller specs don’t include (like c and scheme) such as hash tables and whatnot. the key point is really that what it doesn’t include and that’s a lot of bloat. mind you, bloat is in the eye of the beholder. one man’s bloat is another man’s indespensible feature. that’s why you don’t put everything in the core of a general purpose language. the c spec contains nothing about gui’s, common data structures or even threading.
11. in a way, i guess this goes back to the interactive thing. although, it extends a bit farther. sometimes i just need a function to do something. why do i have to create a class. there is no real class, it’s just a function or multiple functions.
12. this is really an important one. it’s kind of related to being not statically typed. machines are constantly getting faster. these seems like a good trade off in performance for convenience/productivity. why in 2006 do i still have to consider how big a number should be? if i add 2 numbers together, is my result variable big enough. this is ridiculous. not to mention that there have been algorithms for handling big numbers efficiently for quite some time. while there is bignum support in java, it’s not integrated to the base language syntax. you have to use the specific classes. you can’t type “x + 100″, for example. lisp has supported this for quite some time.
13. having types is good. having to specificy what the type of a variable is annoying and generally not necessary. an additional argument towards this is that using unit tests is better because it actually tests your code is correct and not just type consistent. this is still debatable, but i don’t really consider it a big win to have static typing. especially in a high level language.
14. while there isn’t a system specified within the ansi common lisp spec, the apparent winner appears to be asdf. lisp also doesn’t require you to split every class into a separate file, like java, so for a library, you can include all code into a single file. there is support for modules, similar to java packages. there is a related package called asdf-install that provides for downloading tar.gz files from the internet and installing them. this may be usable in replacing ‘java -jar’ as mentioned in that bullet.
15. this is the one bullet where lisp falls down. starting a program is largely implementation dependent. you can always load the file into the interpreter and call your main function. however, a jar type system does not exist. however, it could be written in a implementation neutral way. some implementations also provide for generating a full native binary that can execute without a lisp runtime (well, it’s embedded into the executable). the native binaries do not provide for storing external resources like images and css files, etc. this is less of an issue with web applications and a general problem for most every language, but java. neither python or ruby currently support such a feature as far as i know. i believe python recently added some sort of zip package support similar to jar, but i haven’t researched it any further. i suppose it could be also questionable if loading the entire jar files into memory and keeping them there is really necessary or desirable. especially for included bloated core libraries.
16. removing grunt work is really where common lisp’s main unique feature comes into play: macros. there’s really nothing in common between lisp macros and c macros. this is really summed up well by the “story of mac” section from “practical common lisp”. the main point is that code generation is a desirable and probably required feature heading forward. instead of having separate stand-alone programs and syntax for generating code, it’s all included in the core language with the same syntax. to me, this is one of the most appealing features of lisp.

i’m sure i haven’t generally done any justice to what can be gathered from “practical common lisp”. there are many additional features that i haven’t even mentioned. for example, the object system is far superior to any other i’ve seen. one interesting aspect is that you can create methods on subclasses and control whether they run before, after or around the superclass method, as well as combining methods in the class hierarchy in defined order. i’ve wished for something this on multiple occasions.

what’s the catch?

i’m not really sure there is one. the main one would probably be manager types pointing out that it’s impossible to find replacement staff. yeah, like the replacement staff for other languages is usually so good. a much better strategy for normal businesses is to higher software companies to build software with a support contract. that companies is suspected to keep staff and can probably do a better job of keeping quality programmers than some asp/.net programmer at . if you have interesting problems they will attract good programmers who will not want to leave. if they do leave, you have an interesting problem that will attract good programmers. besides, lisp seemed uniquely qualified (through macros) to creating good frameworks for creating applications that may not require the best programmer in the world. there’s basicall zero syntax to learn and everything else could be a domain specific language which may not even require a programmer. i suspect it’s much harder to hire someone familiar with a specific problem domain than to hire a programmer. getting both is probably near impossible.

i don’t consider this issue a problem. i’m considering what makes me happy and more productive.

real issues may fall into the category of not having networking or threading/multiprocessing in the standard. c doesn’t either. these feature are included in implementations and there are multiple freedom software implementations that include these features. sbcl appears to be the best since it supports native threads on linux. it also apparently has a windows port, but i don’t think it uses native threads yet (which means not using multiple cpus).

conclusion

i’m moving forward with common lisp (using sbcl, there’s a blog posting a comparison of open source lisp implementations. i think the grid could provide a more general commentary of software support. you want the most oses use x86. you want the most platforms use linux.) going forward. my ever so brief flirtation with python is done. i’m trading whitespace significants for a bunch of parentheses (which really doesn’t seem to be a problem with good syntax aware editors).

common lisp on debian and therefore ubuntu is well integrated and packaged. a lot of libraries available.

it also appears that i’ll have to get used to using emacs since slime seems to be a decent development environment. emacs is much more usable with the control key in the correct location (where the wasteful caps-lock key is on pc keyboards). i’m not generally a fan of ide type things, but slime seems alright. particularly easily bringing up the ansi spec in a browser. it also has tab-completion of know variables and other symbols.

i really don’t think i’ve made that good of an argument, but i’ve run out of steam. this probably should be a page instead of a post, but i’d like to get on with my life and do useful things. maybe one day.

screw it, i’m done.