As an ArgoUML contributor I'm going to blog my activities here, so that they may draw interest by other developers or help other developers when doing tasks similar to what I've done. AND(!) the grand vision that makes an Argonaut what he is, TO THRIVE IN THE BIG DANGEROUS WORLD, TAKING THE Argo TO A GOOD SHORE ;-))

Wednesday, May 12, 2010

3rd European Lisp Symposium – a light review

The symposium was nice, small enough that you were capable of getting acquainted with a large percentage of persons and individually chatting with the presenters after their talks. There were a majority of persons related to academia, but, due to the large participation of SISCOG, I think that industry was almost half the number of persons. It is funny that by the number of academia persons is clearly larger from the point of view of presentations and involvement in the organization. We need more industry involvement not only for paying the checks, but, also for presenting and participating in the organization.

I would like to have seen some specific content related to Clojure. Alas, I should have proposed a lightning talk, but, I still don't have anything to show yet, so, the solution would be to spend one night preparing things, but, I preferred to enjoy the dinners. There is something that should have been added related to cloud computing and web applications. Note that RavenPack although not being that, it is a real time data service for financial companies, so, it isn't a mainstream application, but it is very interesting on its own and it is novel.

The overall feeling which I get of the Symposium is that of a group of persons who still suffer at the fact that being Lisp such a special thing, the Symposium should have a larger number of persons, the companies that use Lisp should be larger and more profitable, the open source libraries more polished and more in number, etc. I think that we like the persons which are part of the community and the accessibility of our stars, such as Kent Pitman. We don't like the fact that Lisp is still foreign in large swathes of the industry. Nevertheless, we are seeing a resurgence of Lisp and its use both in academia and in the industry. Now we need to get scale and solutions, so, like for other languages, we need some killer applications / frameworks. This might be a multitude of them, since there are several Lisp dialects and each one may pursuit something different.

Friday, May 07, 2010

3rd European Lisp Symposium, day 2

Tutorial: Parallel Programming in Common Lisp by Pascal Costanza

09:00 - 10:30

Parallel programming is the wave of the future: It becomes harder and harder to increase the speed of single-core processors, therefore chip vendors have turned to multi-core processors to provide more computing power. However, parallel programming is in principle very hard since it introduces the potential for a combinatorial explosion of the program state space. Therefore, we need different programming models to reduce the complexity induced by concurrency.

Common Lisp implementations have started to provide low-level symmetric multi-processing (SMP) facilities for current multi-core processors. In this tutorial, we will learn about important parallel programming concepts, what impact concurrency has on our intuitions about program efficiency, what low-level features are provided by current Common Lisp implementations, how they can be used to build high-level concepts, and what concepts Lispers should watch out for in the near future. The tutorial will cover basic concepts such as task parallelism, data parallelism and pipeline models; synchronization primitives ranging from compare-and-swap, over locks and software transactional memory, to mailboxes and barriers; integration with Lisp-specific concepts, such as special variables; and last but not least some rules of thumb for writing parallel programs.

QUESTIONS

I questioned Pascal about the portability of parallel code and there is none up-to-now. This smells like an opportunity.

On a question of mine about the need to sync the functional data structures internally - for instance FSet is portable - I got a book reference: Purely functional data structures - it was suggested to me that I should read this to understand why there is no need for synchronization. THOUGHT: my lack of an academic background in computer science is to blame for my lack of knowledge about things like this, even after having read about it in respect of the Clojure data structures.

Session III (The Language)

11:00 - 12:45

CLoX: Common Lisp Objects for XEmacs by Didier Verna

Debugging CLoX is a living hell.

Didier made an excellent and very dynamic presentation. There should be no problem in moving what he got up-to-now into GNU Emacs. He intends to optimize some parts to C, though, and then it would be a problem, because XEmacs C level is incompatible with GNU Emacs C.

CLWEB: A literate programming system for Common Lisp by Alexander Plotnick

CLWEB is language specific with advantages for Common Lispers. No tracking back to the source file position yet. Complexity due to the different objectives of the CLWEB reader and Common Lisp reader. Common Lisp STREAM libraries and pretty printer are great! Code walker for Common Lisp - based on the one from the macro-expand-all paper(?). It would be nice if there was more standard ways to interact with the Lisp operators, as to have an easier time creating a walker. Works in SBCL, Allegro and another (Clozure ?).

Keynote: Lots of Languages, Tons of Types by Matthias Felleisen, Northeastern University

Since 1995 my research team (PLT) and I have been working on a language for creating programming languages - small and large. Our code base includes a range of languages, and others contribute additional languages on a regular basis. PLT programmers don't hesitate to pick our lazy dialect to implement one module and to link it to a strict language for another module in the same system. Later they may even migrate one of the modules to the typed variant during some maintenance task.

An expressive macro system is one key to this riches of languages. Starting with the 1986 introduction of hygienic macros, the SCHEME world has worked on turning macros into tools for creating proper abstractions. The first part of my talk will briefly describe this world of modern macros and its key attributes: hygiene, referential transparency, modularity of macros, phase separation, and macro specification.

The second part of my talk will focus on how to equip LISP-like languages with a sound type systems and that will illustrate the second key idea, namely, monitoring the interactions between different languages. Our approach to type systems allows programmers to stick to their favorite LISP idioms. It mostly suffices to annotate functions and structures with type declarations during maintenance work. To ensure the soundness of this information even when higher-order values flow back and forth between typed and untyped modules, module boundaries are automatically equipped with software contracts that enforce type-invariants at all levels.

PLT Scheme is now Racket. Types are very important for maintenance work.

Type theory is stupid. Design driven to make typed Lisp programming practical. Combining the macro system and the type system is still an unresolved challenge.

Panel – does Lisp matters?

I'm part of the panel and therefore instead of notes the following is what I prepared to say. Eventually the specifics weren't followed...

My small intervention notes

  1. Name
  2. disclaimer: less Lisp Experience than the majority of the persons in the room. Although I'm a SISCOG employee, what I will say represents my opinions and I don't know if they are aligned with the ones of SISCOG administration or with the ones of the majority of SISCOG employees.
  3. background:
    1. Physics Engineering, some experience in software development both for data processing and analysis and for scientific instruments
    2. then I started working as a software engineer in telecommunications domain, specifically, in the management of telecommunications networks
    3. about 3 years ago I joined SISCOG and started working in the railway planning and management domain and yes, programming in Common Lisp.

So, SISCOG was founded by 2 professors of the IST - an engineering academical institution. They wanted to use Artificial Intelligence and in the 80s that meant Lisp.

Question: Does Lisp matters?

Yes it does, because of the following three things:

  • Inspiration – it has been an inspiration to lots of persons and languages. This is nothing new, but, many of the new languages still steal from Lisps. And normally they are still crippled copies. So Lisp matters even if it didn't existed anymore, although with open source, as Kent Pitman said yesterday, it isn't possible to kill it.
  • Edge – it provides an edge for its users. Be it due to its dynamic nature, macros, tools, history, smart people magnet, etc.
  • Research – Lisp is very important due to its flexibility and ease of experimenting with new ideas. I will just refer as an example to the previous keynote by Matthias Felleisen.

Quote by Matthias Felleisen:

we are the cockroaches of the world, nobody can kill us!

Announcements, Symposium wrap-up

Didier Verna announced that the 4th European Lisp Symposium will be next year in Hamburg, Germany, on March 30 and April 1. Deadline to submissions will be around the end of the year. This time around I will try to have something to present. Theme parallelism and ...

Thursday, May 06, 2010

3rd European Lisp Symposium, day 1

I'm participating in the 3rd European Lisp Symposium. My employer, SISCOG is sponsoring my participation and of a bunch of others, like the other Luís Oliveira of CFFI fame.

What follows are my notes of day one of the 3rd ELS, with an organization which follows the programme.

Welcome

Wifi not working...

The TI Explorer Lisp Machine. It is possible to submit ideas for lightning talks.

Keynote: Going Meta: Reflections on Lisp, Past and Future by Kent Pitman, HyperMeta Inc.

Over a period of several decades, I have had the good fortune to witness and influence the design, evolution, standardization and use of quite a number of dialects of Lisp, including MACLISP, T, Scheme, Zetalisp, Common Lisp, and ISLISP. I will offer reflections, from a personal point of view, about what enduring lessons I have learned through this long involvement.

Both the programming world and the real world it serves have changed a lot in that time. Some issues that faced Lisp in the past no longer matter, while others matter more than ever. I'll assess the state of Lisp today, what challenges it faces, what pitfalls it needs to avoid, and what Lisp's role might and should be in the future of languages, of programming, and of humanity.

Maybe Kent would collaborate in data standards and not language standards – the objective is to have more choice when selecting what programming language and OS you use to do your work...

Interesting, during the AI Winter, due to the risk of Lisp failing altogether (vendors were near to bankruptcy) there was some need to control the news to be more positive about Lisp. Then Gabriel presented Worse is Better and plenty of persons got mad at him.

Quote of a saying someone had at Symbolics:

At Symbolics, we make hard problems easy and easy problems harder.

Lisp has became the nerd of the programming languages. Lisp became detached. At that time Lisp was different and that was very relevant.

Book references: You Are Not a Gadget by Jaron Lanier; Crossing the Chasm

Some people like to be guided, actually they would pay to be guided. Smart people often don't get this.

The effects of free software: languages can no longer be killed.

The effects of the internet: Lisp being used in the server hidden.

The promise of Clojure: multi-core, it is connected (being in the JVM), vision (Rich Hickey).

QUESTIONS

Why is Lisp the language and a Runtime? Why isn't it just the language? One potential problem is CLOS, maybe we need to drop it and actually try to move in a way that it becomes more attached.

Session I (Mathematical Applications)

Verifying monadic second order graph properties with tree automata by Bruno Courcelle and Irne Anne Durand

Hard to understand and I didn't followed it fully...

A DSEL for Computational Category Theory by Aleksandar Bakic

A Software Engineer that worked on this as a hobby. He used CLOS and functions, trying to mix the two to improve the state of the art of the DSL design for this application, based on what was achieved in the other DSLs based in more functional languages. Book reference, the book uses Standard ML.

Keynote: Reading the News with Common Lisp by Jason Cornez, RavenPack

The financial industry thrives on data: oceans of historical archives and rivers of low-latency, real-time feeds. If you can know more, know sooner,or know differently, then there is the opportunity to exploit this knowledge and make money. Today's automated trading systems consume this data and make unassisted decisions to do just that. But even though almost every trader will tell you that news is an important input into their trading decisions, most automated systems today are completely unaware of the news - some data is missing. What technology is being used to change all this and make news available as analytic data to meet the aggressive demands of the financial industry?

For around seven years now, RavenPack has been using Common Lisp as the core technology to solve problems and create opportunities for the financial industry. We have a revenue-generating business model where we sell News Analytics - factual and sentiment data extracted from unstructured, textual news. In this talk, I'll describe the RavenPack software architecture with special focus on how Lisp plays a critical role in our technology platform, and hopefully in our success. I hope to touch upon why we at RavenPack love Lisp, some challenges we face when using Lisp, and perhaps even some principles of successful software engineering.

CTO of RavenPack. No Lisp code in the presentation, but, he is passionate about Lisp and software engineering and still writes plenty of code.

Quants traders. The opportunity - News are important, but, the algorithms didn't used it. News metadata is unreliable. 27000 companies tracked. Manage the historical news database - entity detection and time dependent data.

They use Lisp because they like Lisp and not because it was a problem that only Lisp can solve. Their customers don't care about RavenPack using Lisp. They also don't sell software, they sell data.

They worked with Franz to improve the interface to Oracle DB. Although they were acquainted of some solutions like CL-SQL, it didn't supported some things they wanted, like stored procedures. Other tools being used are: Linux, Git, Windows, Jakarta HTTP client, Java for integration, Apache Server, etc.

Common Lisp technologies: CL-XMPP, Jlinker (Franz), Allegro Server (Franz), Lisp Server pages, Salza compression, CL-store, etc.

Techniques:

  • Bayesian filters – human experts that classify news as Ham or Spam which serves as training data. Accuracy of around 68%.
  • Restricted Boltzmann machine – compresses some complex full representation into its minimal representation. Accuracy around 87%. It isn't enough for quanta trading.
  • Processing Headlines, based on templates or patterns. Kind of rule processing.

Software architecture: boxes for receiving data, boxes for emitting data, boxes for analysis of historical data. Oracle at the center.

Collectors (read news) -> Real-time Analysis -> Event Server -> 
 Lisp                       Lisp + CL-Store        CL-XMPP (chat rooms)

Lisp frustrations: GC (Lisp is behind the curve compared with JVM and CLR), handling of out of memory errors, weak library support for HTTP, Lisp as a scripting language it is chatty on *standard-out* and last but not the least a lot of unfinished libraries.

Lisp Joy: patch live systems, optional keyword parameters, tools (debugger, REPL, tracer, inspector), FFI and macros.

Book reference: Coders at Work by Peter Siebel.

Coffee

15:30 - 16:00

I was indirectly invited via Tiago Maduro Dias to participate in the final panel of 3rd-ELS. After about 1 minute considering if I was the appropriate person to represent SISCOG, I said to myself, yeah, I'll do it, it will be a honor, let me see if I'm capable of it!

Session II (The Outside World)

16:00 - 18:00

Marrying Common Lisp to Java, and Their Offspring by Jerry Boetje and Steven Melcher

How Lisp looks in Java. Very detailed explanation on how Common Lisp is implemented in Java in CLforJava.

THOUGHTS: I would like that there was a similar design overview for ABCL!...

Tutorial: Computer Vision with Allegro Common Lisp and the VIGRA Library using VIGRACL Benjamin Seppke and Leonie Dreschler-Fischer

Offers filters, morphological operations, feature detectors, segmentation, transformations... All at the comfort of an Allegro REPL near you :-)

VIGRA Library - no Google, its not Viagra! C++, template based, unit tested, multi-platform and which is fairly complete in terms of available algorithms.

VIGRACL uses defsystem. It has a multilayer design with 4 layers. In the bottom, there is the VIGRA library in C++, then a C wrapper, then Allegro Common Lisp FFI and finally an abstraction layer implemented in Common Lisp on the top.

Design: multi-layer, slide.

Saturday, May 01, 2010

SISCOG is hiring developers

SISCOG is hiring for our projects department. If you want to program in Common Lisp professionally, are interested in the railway domain (planning and management) and want to be my colleague ;-), apply please.

Reader Shared items

Followers