As an ArgoUML contributor I'm going to blog my activities here, so that they may draw interest by other developers or help other developers when doing tasks similar to what I've done. AND(!) the grand vision that makes an Argonaut what he is, TO THRIVE IN THE BIG DANGEROUS WORLD, TAKING THE Argo TO A GOOD SHORE ;-))

Wednesday, December 30, 2009

First experiments with ArgoUML's UML2 implementation

To answer Thomas' question regarding ditching *.profile file in ArgoUML persisted projects, I started experimenting with the UML2 implementation that is currently being worked on. The problem is that I don't know it well enough to understand if it is possible to do without the file, so, I thought, well, I should experiment with the implementation and start baby stepping the ArgoUML UML Profile for C++ and the C++ module in the path to be UML2 capable.

But, here comes the pain of experimenting with a not production ready implementation – the persistence of the introduced profile projects isn't working correctly for me. Basically, it isn't loading correctly profiles which were persisted. Not so basically, when you load a *.zargo file which model (*.xmi file contained) is a profile, the case is handled exceptionally, but, correctly in org.argouml.persistence.UmlFilePersister.doLoad(File, File, ProgressMgr) (see UmlFilePersister.java:388). But, then, contrary to what I would expect, the org.argouml.persistence.ModelMemberFilePersister.readModels(InputSource) (see ModelMemberFilePersister.java:258), fails to get hold of a list of UML elements with some contents.

XMI format error : org.argouml.model.UmlException: java.io.IOException: Tag uml:Profile not found
If this file was produced by a tool other than ArgoUML, please
check to make sure that the file is in a supported format, including
both UML and XMI versions.

If you believe that the file is legal UML/XMI and should have loaded
or if it was produced by any version of ArgoUML, please report
the problem as a bug by going to http://argouml.tigris.org/project_bugs.html.
 System Info:
ArgoUML version : PRE-0.29.4
Java Version : 1.6.0_16
Java Vendor : Sun Microsystems Inc.
Java Vendor URL : http://java.sun.com/
Java Home Directory : /usr/lib/jvm/java-6-sun-1.6.0.16/jre
Java Classpath : /home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/build-eclipse-tests:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/build-eclipse:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-infra/lib/log4j-1.2.6.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model/build-eclipse:/home/luis/programas/eclipse-ganymede/plugins/org.junit_3.8.2.v20080602-1318/junit.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/lib/commons-logging-1.0.2.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/lib/gef-0.13.1M3.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/lib/ocl-argo-1.1.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/lib/swidgets-0.1.4.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/lib/toolbar-1.4.1-20071227.jar:/home/luis/programacao/ArgoUML/svnco/argouml/tools/lib/easymock12.jar:/home/luis/programacao/ArgoUML/svnco/argouml/tools/jdepend-2.9/lib/jdepend-2.9.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-mdr/lib/jmi.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-mdr/lib/jmiutils.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-mdr/lib/mdrapi.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-mdr/lib/mof.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-mdr/lib/nbmdr.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-mdr/lib/openide-util.jar:/argouml-core-tools/build-eclipse:/home/luis/programacao/ArgoUML/svnco/argouml/tools/junit-3.8.2/junit.jar:/home/luis/programacao/ArgoUML/svnco/argouml/tools/apache-ant-1.7.0/lib/ant.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app/lib/antlr-2.7.7.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.common_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.ecore_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.ecore.change_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.ecore.change.edit_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.ecore.edit_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.ecore.xmi_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.edit_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.mapping_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/uml2/org.eclipse.uml2.common_1.5.0.v200905041045.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/uml2/org.eclipse.uml2.common.edit_1.5.0.v200905041045.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/uml2/org.eclipse.uml2.uml_3.0.1.v200908281330.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/uml2/org.eclipse.uml2.uml.edit_3.0.0.v200905041045.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/uml2/org.eclipse.uml2.uml.resources_3.0.0.v200906011111.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/lib/emf/org.eclipse.emf.mapping.ecore2xml_2.5.0.v200906151043.jar:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-model-euml/build-eclipse:/argouml-core-infra/bin:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-diagrams-sequence2/build-eclipse:/home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-core-umlpropertypanels/build-eclipse
Operation System : Linux, Version 2.6.29-1-netbook
Architecture : i386
User Name : luis
User Home Directory : /home/luis
Current Directory : /home/luis/programacao/ArgoUML/svnco/argouml/src/argouml-app
JVM Total Memory : 23228416
JVM Free Memory : 5806928


 Error occurred at : Wed Dec 30 08:05:01 WET 2009
Cause : org.argouml.model.UmlException: java.io.IOException: Tag uml:Profile not found
at org.argouml.model.euml.XmiReaderEUMLImpl.parse(XmiReaderEUMLImpl.java:162)
at org.argouml.persistence.ModelMemberFilePersister.readModels(ModelMemberFilePersister.java:258)
at org.argouml.persistence.ModelMemberFilePersister.load(ModelMemberFilePersister.java:112)
at org.argouml.persistence.UmlFilePersister.doLoad(UmlFilePersister.java:389)
at org.argouml.persistence.ZargoFilePersister.doLoad(ZargoFilePersister.java:240)
at org.argouml.ui.ProjectBrowser.loadProject(ProjectBrowser.java:1577)
at org.argouml.ui.LoadSwingWorker.construct(LoadSwingWorker.java:75)
at org.argouml.ui.SwingWorker.doConstruct(SwingWorker.java:139)
at org.argouml.ui.SwingWorker$2.run(SwingWorker.java:267)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Tag uml:Profile not found
at org.argouml.persistence.XmlInputStream.realRead(XmlInputStream.java:378)
at org.argouml.persistence.XmlInputStream.skipToTag(XmlInputStream.java:226)
at org.argouml.persistence.XmlInputStream.read(XmlInputStream.java:136)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2933)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:739)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
at org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.load(XMLLoadImpl.java:181)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doLoad(XMLResourceImpl.java:180)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1494)
at org.argouml.model.euml.XmiReaderEUMLImpl.parse(XmiReaderEUMLImpl.java:160)
... 9 more
-------
Full exception : org.argouml.persistence.XmiFormatException: org.argouml.model.UmlException: java.io.IOException: Tag uml:Profile not found
at org.argouml.persistence.ModelMemberFilePersister.readModels(ModelMemberFilePersister.java:287)
at org.argouml.persistence.ModelMemberFilePersister.load(ModelMemberFilePersister.java:112)
at org.argouml.persistence.UmlFilePersister.doLoad(UmlFilePersister.java:389)
at org.argouml.persistence.ZargoFilePersister.doLoad(ZargoFilePersister.java:240)
at org.argouml.ui.ProjectBrowser.loadProject(ProjectBrowser.java:1577)
at org.argouml.ui.LoadSwingWorker.construct(LoadSwingWorker.java:75)
at org.argouml.ui.SwingWorker.doConstruct(SwingWorker.java:139)
at org.argouml.ui.SwingWorker$2.run(SwingWorker.java:267)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.argouml.model.UmlException: java.io.IOException: Tag uml:Profile not found
at org.argouml.model.euml.XmiReaderEUMLImpl.parse(XmiReaderEUMLImpl.java:162)
at org.argouml.persistence.ModelMemberFilePersister.readModels(ModelMemberFilePersister.java:258)
... 8 more
Caused by: java.io.IOException: Tag uml:Profile not found
at org.argouml.persistence.XmlInputStream.realRead(XmlInputStream.java:378)
at org.argouml.persistence.XmlInputStream.skipToTag(XmlInputStream.java:226)
at org.argouml.persistence.XmlInputStream.read(XmlInputStream.java:136)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2933)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:739)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
at org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.load(XMLLoadImpl.java:181)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doLoad(XMLResourceImpl.java:180)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1494)
at org.argouml.model.euml.XmiReaderEUMLImpl.parse(XmiReaderEUMLImpl.java:160)
... 9 more

I wanted to prepare a UML2 compatible UML profile for C++ to test that we may live without the *.profile files. But, to make this I can't create it from scratch, I need something like a UML1.4 profile to UML2.x profile conversion. Lets try to do it manually...

Using Emacs, I opened my test profile made with ArgoUML UML2 (MyUML2Profile.xmi), the base UML2.2 profile of ArgoUML (default-uml22.xmi), and the UML1.4 profile for C++ (CppUmlProfile.xmi). One thing I noticed is that the XMI format changed very much. Specifically, in the begin of the file, it is much more noisy. If you don't believe my word, check the following in default-uml22.xmi:

  • What the heck is the metaclassReference attribute and why does it have so much noise?
  • It is importing element by element many of the standard UML2.2 metamodel elements. It even does this in duplicate:
      <elementImport xmi:type="uml:ElementImport" xmi:id="_pGIiAMosEd6otN4YcPG1_w" importingNamespace="_PB5O8MoqEd6otN4YcPG1_w">
        <importedElement xmi:type="uml:Class" href="http://schema.omg.org/spec/UML/2.2/uml.xml#Abstraction"/>
      </elementImport>
      (...)
      <elementImport xmi:type="uml:ElementImport" xmi:id="_wcazEMosEd6otN4YcPG1_w" importingNamespace="_PB5O8MoqEd6otN4YcPG1_w">
        <importedElement xmi:type="uml:Class" href="http://schema.omg.org/spec/UML/2.2/uml.xml#Abstraction"/>
      </elementImport>
      
    Does this mean that you have to make this dance twice for it to work?!?

Saturday, December 05, 2009

Design of the ArgoUML's projects recovery tool

I was starting to try to hack my way to implement the ArgoUML's projects recovery tool which I proposed in issue 5862: Support for moved/renamed/merged profiles when I started to hesitate on what to do. So, I did the right thing, went to ArgoUML 0.28.1 and started a lightweight design session.

I decided that the tool should exist as a sub-project of ArgoUML core. For now its use case(s) is very simple:

ArgoUML's projects recovery tool use cases.

Picture 9 – ArgoUML's projects recovery tool use cases.

This is a high level view of the static structure I arrived:

ArgoUML's projects recovery tool core high level static structure.

Picture 10 – ArgoUML's projects recovery tool core high level static structure.

This is the activity diagram for it:

ArgoUML's projects recovery tool core happy path activity diagram.

Picture 11 – ArgoUML's projects recovery tool happy path activity diagram.

Design sence – deep lessons in software design by Michael Feathers

I watched Michael Feathers' talk Design sence – deep lessons in software design, and I what follows are my notes for future reference.

  • Software normally has a characteristc of being driven by power laws.
    • The shape of java – research project that studies the source of both open source proprietary projects in Java.

  • Everything is an object or may be considered as such.
    • Smaltalk – everything is actually and object.
    • Ruby – it goes a long way into almost everything being objects.
    • C and other procedural languages may be considered as object oriented if you consider the whole program as an object.

  • A type is a set of values. Covariance and contravariance.
  • You can't make software more correct with redundancy.
  • Protection is a social problem and not a technical problem.
  • There are no requirements. There is only design.
  • Design by laundry list.
  • Names are provisional.
  • Physical architecture shapes logical architecture.
  • Physical architecture can influence the kind of problems we are able to solve.
  • Conway's law.
  • You can finess error handling with good design.
  • Error policy is as important as the main line.
    • Keep errors at the boundaries or handle errors at the boundaries of the system.
    • Null object pattern.
  • Logging is a first class design consideration.
  • Databases are good for more than storing data.
  • Objects want to be asynchronous.

Friday, November 20, 2009

UML profiles management issues

I'm currently handling two issues related to ArgoUML management of profiles:

Monday, November 16, 2009

Other software ideas

These aren't related to ArgoUML, but, since they are strongly related to software, they'll be registered here.

Emacs Lisp Unit

Build a unit test framework for Emacs Lisp and with that learn more about Emacs Lisp. Upsss, there is already ElUnit and behave.el.

Contribute to jnil

jnil is an open source project by a colleague from SISCOG, Tiago Maduro Dias. Initially I wanted to join some open source Common Lisp project and the prospect of being able to pair program an open source project with a more experienced Common Lisp developer was very interesting. Unfortunately, I didn't want to leave ArgoUML without reaching a feeling of closure, so, I never joined. It is still an interesting idea...

Genetic programming based in the Common Lisp Condition System

Since I read about the idea of evolving software with genetic algorithms, I was fascinated by it. When I really learned about the Common Lisp Condition System and its capability to offer restarts I wandered if it would be possible to combine the two to provide a way to avoid the need to have mutations to generate correct Common Lisp. It might be a good way to make safe boundaries between mutated code and stable code, to avoid infinite loops, deadlocks, etc.

FitNess Common Lisp adapter

Create an adapter for Common Lisp for FitNess, to enable using this acceptance test framework in Common Lisp projects. With ABCL maturing very fast and being well supported, I wonder if this would be difficult at all.

Learn Clojure

Give myself some time to learn Clojure.

Develop organization absences planer in Google App Engine

The process(es) that organizations normally use to plan their collaborators vacations and absences are usually document driven and the verifications and checks of accordance with the rules and legislation manual. I suffer from this at SISCOG and in SIEMENS, although it wasn't document based, it lacked considering the organization needs, like to have a minimum number of persons in a certain development team available at all times in a year. It didn't considered previous history, like a certain person always not getting his vacations in the Christmas season while others always get them.

It would be a perfect opportunity to learn web development techniques in a restrained platform such as Google App Engine, but, possibly using a JVM based language, such as Clojure or Jython. I wanted also to show this at SISCOG, to have a demo-able example of a Cloud based application, which someday SISCOG may want to try with more commercial purposes...

Online git repositories

I started using distributed revision control systems (DRCS) a while ago, first with mercurial (hg), then with Git, when my ex-colleague Tom Schutzer-Weissmann created a system to import the SISCOG repositories from the in-house defined format (CRM/CRI) into Git. Currently the Git repositories are being used by several persons in SISCOG, but, I still haven't been able to proceed with the work enough to consider the proposal of a switch or a pilot.

Currently I'm more familiar with Git, but, my problem is that I'm not cooperating online using one of these DRCSs. So, I wanted to setup a GitHub account and move in there some of my repositories and potentially host there a fork of ArgoUML. By having a Git based ArgoUML fork, I would have a much easier life in working offline with contributions and a much nicer sandbox to experiment ideas and patches. This I know for sure, because I have been using the branching features of Git in SISCOG for some time with a huge success and satisfaction.

Finish reading Practical Common Lisp

I stopped reading Practical Common Lisp in the chapter 23. Practical: A Spam Filter. I should give myself some time to finish reading and exercising the book. While at it, I should buy a copy of it, since it is excellent and I would have something physical to borrow to newcomers at SISCOG.

Proceeding with ideas on books to read, On Lisp, by Paul Graham is high on my Lisp related books and several others I placed in my Amazon wish lists.

Two ideas related to ArgoUML

Following with my write-up of ideas to avoid neurological garbage collection before delving into ArgoUML for some time, follows a list of ideas related to ArgoUML, in which I won't elaborate as much as in the entry about medeia.

abstracter

The idea is to register in the background decisions that the ArgoUML modeler does, so that we may deduce from his actions a set of steps that might be used to solve similar problems. This is similar to a refactoring system that my colleague Rui Patrocínio explained to me. This system, takes a set of programmer actions and may apply them to a similar problem upon a programmer request.

This is called abstracter, because the goal would be to give such a tool the reverse engineering information from a software project and let it abstract away the details, creating easy to use diagrams and perspectives about the software. Kind-of something like pointing it to the OpenOffice.org source and letting it create some meaningful abstractions of it. The part where it is learning or being corrected by a developer is interesting, because it would allow experimentation with machine learning principles...

ArgoUML Lisp module

Implement a Lisp language module for ArgoUML. Probably this could have dialect specializations (Common Lisp, Scheme, Clojure, etc), but, have a reusable base. My idea is that this should be implemented in the specific languages and not in Java, in order to more easily receive contributions from developers that are primarily interested in the module.

Note: Rui Patrocínio referred a related project he was considering to do which is a static analyzer for Common Lisp so that it would be possible to support code browsing and refactoring.

The reverse engineering of Common Lisp, Scheme and Clojure should be based in the same communication protocol as the one used by SLIME, which allows things like running APROPOS.

Sunday, November 15, 2009

medeia

From all the ideas my idiot's head has had in the past few years, one has been buzzing very strongly and that is medeia. The name Medea comes from the Argonautica book, so, it is related to ArgoUML and it is the name of a woman that helps the Argonauts and Jason in particular to get the Golden Fleece. She knows one thing or two about magic and uses it. Medeia is Medea in Portuguese related to mediate. Well, the name points a bit to what the idea is about...

medeia is a good name because it is similar to idea in Portuguese, which is written ideia.

As the responsible and main contributor to the ArgoUML C++ module, in the past 5 years, I've seen that this – making a language module – is a hard task. There is a lot that must be considered to provide production ready code generator, reverse engineering, notation and round the horn language module(s). Normally the effort is single handed and when you reach the end, there will always be loopholes and needs to extend it. But the worst part is how much of reinventing the wheel goes into it. Lets put the things in a clear way, by enumerating each part of a typical language support module:

  • UML Profile for the language:
    • You need a core language profile with stereotypes that may be used for tagging UML model elements, in order to apply tagged values to the model elements.
    • To use the profile in a programmatic way you need some wrapping classes, such as the class ProfileCpp in the C++ module.
    • Potentially you would also want to place in the module UML profiles for libraries that are used in plenty of projects that use the language – in the case of C++ we may have boost.

  • Language generator:
    • Use of a template engine for formatting the generated code. (For an example see StringTemplate.)
    • A code generator, which maps the UML model into an Abstract Syntax Tree (AST) which would be handed over to the template engine.
    • A way not to trash the code that already exist, like the current protected sections in the C++ generator.
    • Some form of mapping the documentation in the UML model into the language standard(s) format(s). And you need this to be customizable because projects typically develop specific variants of the documentation standards or there is some company standard that they must use. I would say that this fits more or less in the same problem/solution as the code generator being made by two parts – a mapping from UML to an AST and a template engine which is fed by the AST – but, I think that here there might be more needs for giving customization options for enhancing the AST generator.
    • Potentially there is the need to create code from specific UML diagrams or the model elements beneath specific diagrams. For instance, generating the classes that implement the state design pattern for a state machine (check also this nice article by Robert C. Martin (aka Uncle Bob) on finite state machines and UML.
      As a side note, you may not need to do this if you use the ArgoUML Pattern-Wizard module.

  • Language reverse engineering:
    • There must be a language grammar, typically, in ArgoUML, this is based in ANTLR and it is common that there is something for your target language.
    • With the language grammar you can generate a Parser, it might even be possible that an AST walker is generated for you, but, your mileage may vary and the available Parser and AST walker might not be fully customized to your needs. For instance, I was capable of getting a ANTLR C++ grammar ported to Java, but, this wasn't prepared to make an AST and I failed to change it into one. It has some bugs which up-to-now I didn't solve. Finally, I'm the one that must keep it evolving with the original grammar which was made for C++. Even the original grammar is now unsupported, which is problematic with the evolution of the C++ standard to C++ 0x.
    • Even with the AST walker there is the need to perform the mapping from language constructs to UML model elements. This mapping must be performed in a way that doesn't create duplicated model elements when the same source files are imported multiple times.
    • The grammars are normally created for blue sky situations where you have all the information, the source code is well formed, etc, etc. The reality is that the reverse engineering modules must be designed to handle malformed sources and work with incomplete information – i.e., partial imports.
    • In C++ you need a C preprocessor. Fortunately there is one open source C preprocessor implemented in Java which the C++ module now uses.
    • Users normally aren't satisfied by simple import and they normally want that the reverse engineering creates some simple UML diagrams from code. The Java module has this.
    • Finally, the killer feature of a UML reverse engineering module is for a user to point it to a build definition file – for instance a make file for C++ or an Ant build.xml file – and the reverse engineering module would interpret it and import the underlying project fully into a UML model.

  • Language notation – this enables the user to edit UML with the language notation.
    • There is a simple way to create a read-only notation for the language, which is basically to use the code generator in a partial way. That is what the C++ module does in the notation package.
    • The complex part is to create a parser for the notation. The best example is to check the parser that Michiel van der Wulp developed for UML notation. Neither C++ nor Java have a notation parser. If there is such a parser for the language notation it would enable fully editable UML models in the programming language.

  • Round the horn engineering for the language.
    • This is the Lucy in the sky with diamonds scenario. You edit the model and say update code and the code is magically updated without trashing the existing code and potentially adapting the dependent code to the changes you made. Even better, it would understand enough of the build definition to add new files to it or remove unused ones.
      My knowledge about tools that support this is that Together did a good job and that Rational Rose for M$ VC++ did a bad job at it. It is a long time since I tried tools such as these in a professional setting.

  • Language module settings user interface – finally, for each of these things you need to consider how the user is going to interact with it and configure it to his needs.
    • A nice GUI for some things, like the generator options concerning the mapping from UML model elements into the language constructs (e.g., see SettingsTabCpp, but, this has some mixture of several parts of the generator).
    • The reverse engineering also has a GUI which is placed in the dialog that is shown to the user when importing code. Currently the C++ reverse engineering module has no such GUI, but, you can see an example in the Java module – JavaImportSettings.
    • Then there are parts of the UI that may have the form of a template file, but, this is UI and, as such, the template engine must deal with the potential errors introduced by changes the users did.
    • All of this and more must somehow be documented in the user manual, which potentially is translated into other languages...

Description of medeia

So, enough of complaining myself about the fun, but, hard and difficult to finish work of developing language support for ArgoUML in the classic way. Lets now delve into what medeia is!

The medeia idea came to me a long time ago, but, in a gradual way. First I thought about a different solution to develop the reveng C++ module, which was to base it in the reuse of libraries made by the Eclipse CDT project. Then, the obvious stroke my mind and I thought of this as a potential solution for several other language modules, just as the Eclipse guys add support for more and more languages, the ArgoUML could grab it and reuse it. Recently, with the evolution of polyglot software development and the dynamic language boom, both Eclipse and NetBeans are adding support for multiple and interesting languages.

Then, with the Argoeclipse project, I thought that this would be a perfect opportunity to revisit this, but, instead of reusing libraries by placing them in the ArgoUML modules, we could just mediate the things that the existing Eclipse environment knows about the projects the user has in his Eclipse workspace and map those into ArgoUML and back, reusing for this the whole infrastructure that Eclipse has. Note that this is also possible for an eventual Argonetbeans.

Finally, while working at SISCOG and understanding the way Emacs connects via SLIME or ELI to a live Lisp and how that enables you to discover on a live and fully consistent software program about its static structure, I came to the conclusion that medeia could be a bit different for languages that have reflexion built-in (Java included)! You just need to place a little piece of software within the program you want to know about and inquire it from the ArgoUML side. No need to parse source files, understand build systems, etc, etc. Well, you still have some work for generating code and to provide a language notation, but, the medeia idea solves a large part of the problem of reverse engineering modules for a language. It creates some new, though...

Classical reveng module medeia reveng module
Language grammar, AST walker, parser, etc Interface with the IDE (for languages that don't have powerful enough reflexion facilities, such as C++, C, Fortran and etc) or with the live program (for languages that do have such reflexion facilities), that communicates with the ArgoUML module or with Argoeclipse or with Argonetbeans or with Argowhatever.
Mapping from language to UML. Ditto, also needed here.
Robust software to deal with incomplete information or malformed source code. Software that must support some way to filter the information that matters, maybe by allowing the user to specify that the import must be from specific packages or / and by allowing to get the information by specifying incomplete names – similar to IDEs' find type.
C preprocessor or similar things, largely dependent on language quirks. Nothing like that needed on this side, thanks :-)
UML diagrams auto-magically created by the import sources process. Ditto in here too, but, with the difference that probably the user might need to ask for diagram creation and update or offer some settings and try to do the correct action for different user actions.
Understanding build definitions to import full projects with the minimum work by the user. No need for this just for reverse engineering, but, in complex multiple-project scenarios, maybe the users want that the correct UML elements go into the correct UML models... The filtering described above might be a good way for certain languages, but, for languages that simply support namespaces or that are used in a flat package way, like Lisp, this wouldn't work so nicely.

Do you like this idea? Have you seen it applied to other UML tools or to solve similar problems? I recall that this is already used in the context of Emacs and Lisp / Scheme / Clojure, with the SLIME project or others similar to it. But for a UML tool you normally go the way of the classical reverse engineering design described above. Well, maybe Together worked this way when integrated within Eclipse...

Wednesday, November 04, 2009

Dispersion problem – why am I contributing so little to ArgoUML?...

At SISCOG some of the employees – I included – attended a leadership workshop that lasted 3 days and was distributed by February, March and April of 2009. It was very interesting and useful for the development of my soft skills, which I need as part of my role as a team leader. The person giving the workshop was Teresa Roseta of MQI, whom I strongly recommend. SISCOG developed the leadership principles and values in the workshop.

One of the things that was referred there was the need for a person to know himself better. This because it seams that normally good, natural leaders do know themselves very well. One exercise we did for each of us was our professional coat of arms. It is a bit like a personal SWOT analysis, but, the idea was to minimize the number of things that go into each quadrant and to have a middle row with your motto. The things that go into the quadrants are different too. To make a long story short here is my professional coat of arms (the words in [emphasis] are just legends for the contents):

[Strength] easy learning and appreciation for knowledge [Dream] to fulfill one or more remarkable ideas (hint: MEDEIA)
[Motto] do things right (i.e., well/correctly)
[Vulnerability] dispersion [Facilitator] persistence

My vulnerability is dispersion. This causes some problems for my average rate of contributions for anything that isn't my primary activity and currently for my contributions to ArgoUML, which are too few and too slow. Issue 5864 illustrates this. I was given a patch, so, I looked into it and thought, OK, lets help this person scratch his hitch and maybe ArgoUML can get a new developer aboard. Then a difficulty came and I feared that it was something that was related to a part of the code that I never delved into and I placed it on hold until I had enough energy for it. But, in the meantime I had enough energy to do a Programming Praxis problem, to start developing a friendly user interface for htmled, to start a crazy attempt to update all the software in my old Windows XP laptop and to investigate the possibility to move some personal Subversion repositories into Git! At least these are the significant things I remember about between 7th and 30th of October. Why didn't I finished the thing I started in the first place? Dispersion!

I disperse because I'm the kind of person that loves to learn and I have lots of ideas. These ideas are all very interesting for me, but, they take some time to look into, develop and document. They are mostly related to software development, but, they don't intersect as much as possible for them to be developed together or in order for some parts to add to the others. So, these things start conflicting with each other and I don't get much done of anything unless at my work, where I managed to finish some significant things.

I like ArgoUML. The developers there aren't the greatest community ever, but, they are nice persons who care most about making ArgoUML incrementally better and some are pretty decent software developers with high commitment level. I think that I contributed some important work for the C++ module and did some nice work in the UML profile support. But, I'm involved in ArgoUML since 2004 and the pace of my contributions has been very snail like or even slower. Even worst, both the C++ module and the profile support seam a bit of an half-finished business. I think I know where the problem lies – dispersion!

So, I do I fight this? I do it by setting up a challenge for myself – spend one year focused at contributing to ArgoUML. Forget about the software blogs, software books and exercises. Blog about ideas I have, take some notes, so that they don't get neurologically garbage collected, but, then go back to work. In the end, take some ArgoUML vacations for some undetermined time, look at one of these ideas and pick it up if I want to give it a try. (Probably I will want to do something related to my work at SISCOG and if so, it may take one year or more to finish, but, I don't know and for now don't care what that will be.)

So, my aim is mainly to move profile support in ArgoUML up into something that looks a lot like finished business and try to do the same thing for the C++ module (this one is harder, though).

But first, as a last bit of devilish concession to my dispersion I'll spend some time writing and publishing some of the significant ideas in either Argonaut's life if they are ArgoUML or software development related, or in Idiota if they aren't.

Oh, and I have an idea for a trick that will help me not to get distracted. I'm going to get myself an ArgoUML bracelet, just like Uncle Bob has his green wrist band to remember himself that code must be clean and tests green. Maybe I should get a green ArgoUML wrist band!

Thursday, October 22, 2009

Solution for Programming Praxis' Beautiful Code

The following is my solution to the Programming Praxis' Beautiful Code problem in Common Lisp. (The final code is also in pastebin.com.)

I started from a solution in which I ported the code from C to Common Lisp as directly as possible. For instance, the code for the matchhere function was this mess:

;;; matchhere: search for re at beginning of text
(defun matchhere (re text)
  (if (= 0 (length re))
      1
      (if (and (< 1 (length re)) (char= #\* (elt re 1)))
   (matchstar (elt re 0) (subseq re 2) text)
   (if (and (char= #\$ (elt re 0)) (= 1 (length re)))
       (= 0 (length text))
       (when (and (< 0 (length text))
    (or (char= #\. (elt re 0)) (char= (elt re 0) (elt text 0))))
  (matchhere (subseq re 1) (subseq text 1)))))))

Then I changed the above to use cond and abstracted (= 0 (length seq)) to empty, which resulted in this:

(defun matchhere (re text)
  "search for re at beginning of text"
  (cond ((empty re))
 ((and (< 1 (length re)) (char= #\* (elt re 1)))
  (matchstar (elt re 0) (subseq re 2) text))
 ((and (char= #\$ (elt re 0)) (= 1 (length re)))
  (empty text))
 ((and (not (empty text))
       (or (char= #\. (elt re 0)) (char= (elt re 0) (elt text 0))))
  (matchhere (subseq re 1) (subseq text 1)))))

Finally, I turned the loop based match and matchstar into tail call form. For illustration, this is what matchstar initially was:

;;; matchstar: search for c*re at beginning of text
(defun matchstar (c re text)
  (loop when (matchhere re text) do (return 1)
       while (and (not (equal 0 (length text)))
    (or (char= c (elt text 0)) (char= #\. c)))
       when (not (= 0 (length text))) do (setf text (subseq text 1))))

And this is how it is after the refactoring:

(defun matchstar (c re text)
  "search for c*re at beginning of text"
  (cond ((matchhere re text))
 ((and (not (empty text))
       (or (char= c (elt text 0)) (char= #\. c)))
  (matchstar c re (subseq text 1)))))

All done, I'm pretty happy with the result I have now. It is very similar to the Scheme solution by the Programming Praxis author and it certainly is very Lispy.

Monday, September 21, 2009

Customizing Eeebuntu GNU/Linux 3.0 Standard into a development environment

I documented previously the Eeebuntu GNU/Linux 3.0 Standard installation. My intent now is to document the process to make this into a development environment. Since this is an ongoing process, it is possible that I will make new entries that supplement this one, but, for now I hope that this will make the process repeatable and potentially automated.

An important note is that I used the Synaptic Package Manager instead of the command line apt-get package handling utility, so, the commands I'll place bellow maybe won't lead to the result I currently have. This is nevertheless an attempt to automate the process and for this a command line is always better than a list of steps to be performed in some GUI tool.

Install latest Emacs, which is the basis for the majority of text editing and Lisp development.

 sudo apt-get install emacs-snapshot-gtk emacs-snapshot-el

Several version control systems:

 sudo apt-get install git-core git-doc git-gui git-svn subversion

Common Lisp implementations and support for its development:

 sudo apt-get install cl-swank cl-asdf common-lisp-controller sbcl sbcl-doc sbcl-source slime

Standard C++ development support (maybe I should install boost also).

 sudo apt-get install g++ g++-multilib gdb gdb-doc

Python 2.6 development environment (in the future I will probably install the latest Python 3.x also):

 sudo apt-get install idle-python2.6 python2.6-doc python2.6-examples

For future reference I leave here the log of the day when I picked and installed the tools for software development environment.

Commit Log for Fri Sep 18 00:25:50 2009

Installed the following packages: 
blt (2.4z-4.1) 
cl-asdf (1.111-1) 
cl-swank (1:20080223-2) 
common-lisp-controller (6.17) 
emacs-snapshot (1:20090320-1ubuntu1) 
emacs-snapshot-bin-common (1:20090320-1ubuntu1) 
emacs-snapshot-el (1:20090320-1ubuntu1) 
emacs-snapshot-gtk (1:20090320-1ubuntu1) 
g++ (4:4.3.3-1ubuntu1) 
g++-4.3 (4.3.3-5ubuntu4) 
g++-4.3-multilib (4.3.3-5ubuntu4) 
g++-multilib (4:4.3.3-1ubuntu1) 
gcc-4.3-multilib (4.3.3-5ubuntu4) 
gcc-multilib (4:4.3.3-1ubuntu1) 
gdb-doc (6.8-1) 
git-core (1:1.6.0.4-1ubuntu2) 
git-doc (1:1.6.0.4-1ubuntu2) 
git-gui (1:1.6.0.4-1ubuntu2) 
git-svn (1:1.6.0.4-1ubuntu2) 
gitk (1:1.6.0.4-1ubuntu2) 
glibc-doc (2.9-4ubuntu6) 
glibc-source (2.9-4ubuntu6) 
idle-python2.6 (2.6.2-0ubuntu1) 
lib64gcc1 (1:4.3.3-5ubuntu4) 
lib64gomp1 (4.3.3-5ubuntu4) 
lib64stdc++6 (4.3.3-5ubuntu4) 
libapr1 (1.2.12-5ubuntu0.1) 
libaprutil1 (1.2.12+dfsg-8ubuntu0.3) 
libc6-amd64 (2.9-4ubuntu6) 
libc6-dev-amd64 (2.9-4ubuntu6) 
libdigest-sha1-perl (2.11-2build2) 
liberror-perl (0.17-1) 
libpq5 (8.3.7-1) 
libstdc++6-4.3-dev (4.3.3-5ubuntu4) 
libsvn-perl (1.5.4dfsg1-1ubuntu2.1) 
libsvn1 (1.5.4dfsg1-1ubuntu2.1) 
python-tk (2.6.2-0ubuntu1) 
python2.6-doc (2.6.2-0ubuntu1) 
python2.6-examples (2.6.2-0ubuntu1) 
realpath (1.12) 
sbcl (1:1.0.18.0-2) 
sbcl-doc (1:1.0.18.0-2) 
sbcl-source (1:1.0.18.0-2) 
slime (1:20080223-2) 
subversion (1.5.4dfsg1-1ubuntu2.1) 
tcl (8.4.16-2) 
tcl8.4 (8.4.19-2) 
tcl8.5 (8.5.6-3) 
tk (8.4.16-2) 
tk8.4 (8.4.19-2) 
tk8.5 (8.5.6-3)

I have also some more software that currently is installed in my$HOME:

Installing Eeebuntu GNU/Linux 3.0 Standard on my Asus Eee PC 901

I bought an Asus Eee PC 901 with an Asus customized xandros GNU/Linux distribution back in December 2008. I meant to switch to a more “normal” Debian based GNU/Linux distribution right from the start, but, my initial strategy was to use only the Asus Eee PC 901 to make the switch and I started having difficulties, so, this effort stalled for a couple of months. Now, I successfully made the switch to Eeebuntu GNU/Linux 3.0 Standard (which is based on Ubuntu GNU/Linux 9.04 Jaunty Jackalope, being Ubuntu GNU/Linux based on Debian GNU/Linux), and what follows is the successful recipe I used.

  1. I bought a 4GB Kingston DataTraveler USB stick to place there the Eeebuntu GNU/Linux 3.0 Standard for installation.
  2. In an old Compaq Presario 2500 laptop I used KNOPPIX 6.0 / ADRIANE 1.1 - Live CD (another Debian based GNU/Linux distribution) to:
      p
    1. rewrite the partition table of the USB stick as a single primary partition, using the bundled gParted;
    2. format the USB stick as a FAT16 partition by issuing the command
      sudo mkfs.vfat -F 16 /dev/sda1
      (note that my USB stick was identified as sda, in another situation it may be different).
  3. In the same notebook, using UNetbootin in Windows, I created a boot-able USB stick from the Eeebuntu GNU/Linux 3.0 Standard ISO image.
  4. Finally, start-up the Eee PC 901, pressing Esc key to get into the boot device selection menu and choose the USB stick as the boot device. (Note that I changed the BIOS settings, disabling the fast boot and changing the boot priority order, although in retrospect I suspect this wasn't needed.)
  5. Select the default option in the UNetbootin boot menu (still before Linux image in the USB stick starts or wait some seconds.
  6. After the Eeebuntu GNU/Linux 3.0 Standard ISO image starts, double click the Install shortcut in the desktop.
  7. The installation procedure is very simple, and I only made some complex decisions in the manual partitioning, having the following:
    • /dev/sda – this is the master SDD disk with 3.75 GB of memory and it is faster than the secondary SDD disk. I partitioned it in two as follows:
      • /dev/sda1 – with a size of 2.88 GB contains all top folders (/ mount point), except linux-swap, /usr and /home;
      • /dev/sda2 – ~ 900 MB for linux-swap.
    • /dev/sdb – this is the slave SDD disk with 15 GB of memory. I partitioned it in two as follows:
      • /dev/sdb1 – with a size of 6.22 GB, contains /usr;
      • /dev/sdb2 – with a size of 8.81 GB, contains /home.
    I used ext4 file system for all the partitions (except linux-swap).

In retrospect I think that I would be able to do all the steps in the original Eee PC 901 with xandros, but, originally, in December 2008 I had an 8 GB USB stick for which a FAT16 file system isn't possible. Also, the xandros with which the Asus Eee PC 901 comes is based in an old Debian distribution and I wasn't able to execute UNetbootin in it.

I already made some customizations, like removing some programs and installing additional ones (I'm writing this in emacs-snapshot-gtk) and development libraries, but, I'll document this in a different post.

Friday, September 04, 2009

bash hints – for loops

Today I had to process some files in a similar way in a bash shell. After trying using pipes, I reverted to for loops. First iterating for all files in a directory and extracting its contents (they were *.tar.bz2):

(...)/pcl-book/c23-spam/sa-pub-corpus> ls 
20030228_easy_ham_2.tar.bz2  20030228_hard_ham.tar.bz2  20030228_spam.tar.bz2 
20030228_easy_ham.tar.bz2    20030228_spam_2.tar.bz2
(...)/pcl-book/c23-spam/sa-pub-corpus> for f in *; do tar -xjf $f; done

Then, I discovered that the archives contained a file which I didn't wanted, so, I removed the files (the files were named “cmds”):

(...)/pcl-book/c23-spam/sa-pub-corpus> ls 
20030228_easy_ham_2.tar.bz2  20030228_hard_ham.tar.bz2  20030228_spam.tar.bz2  easy_ham_2/  spam/ 
20030228_easy_ham.tar.bz2    20030228_spam_2.tar.bz2    easy_ham/              hard_ham/    spam_2/
(...)/pcl-book/c23-spam/sa-pub-corpus> for f in `ls --ignore=2003*`; do `rm $f\cmds`; done

Sunday, August 30, 2009

Some sw dev links

Thursday, May 07, 2009

Common Lisp dynamic variables

One thing that Common Lisp has which is very powerfull is its dynamic variables. The following code ilustrates the concept:

CL-USER(124): (defvar *x* "Hello") ; *x* is a dynamic variable*X* 
CL-USER(125): (progn
                (let ((*x* "Ola")) ; establishes a dynamic environment for *x*
                  (format t "~a~%" *x*)
                  (setf *x* "Ciao")
                  (format t "~a~%" *x*)) ; end of the dynamic environment
                (format t "~a~%" *x*)) ; back to the initial dynamic environment
                                       ; *x* assumes its original binding 
Ola 
Ciao 
Hello 
NIL 
CL-USER(126): 

Now if this is used with multiple processing and/or closures you get very powerful ways to separate things and a great amount of flexibility almost for nothing.

Wednesday, May 06, 2009

ArgoUML 0.28

Some time ago ArgoUML 0.28 was released. This is a release that contains some new features and plenty of bug fixes. Please come aboard if you're still on land and give us some feedback via ArgoUML mailing lists, ArgoUML users' wiki band forums or via a bug or enhancement report.

Common Lisp hacking with ABCL

I'm riding on a Linux Eee PC 901. For now I haven't installed a different distribution than the customized Xandros that comes with it... The freaking thing isn't booting from a FAT32 USB that I have...

So, and because the main flash drive is almost full, I don't have GCC to build a more mature Lisp. I'm therefore using Armed Bear Common Lisp (ABCL) for some entertainment hacking like solving Programming Praxis problems and reading Peter Seibel's Practical Common Lisp for a second time. ABCL has some quirkcs, like when I tried to build an SVN ccheckout with Sun's JDK 1.6, it had some compilation errors. Nothing serious, and, after some minutes, I was at the REPL in Emacs. For now I tested it with a basic subset of imperative and functional Lisp and it stood well. I wonder if someday I try to setup a Lisp REPL for ArgoUML with ABCL (check issue 3082: evaluating GEF support for template notation spike)...

Solution for Programming Praxis' Rail-Fence Cipher

I found Programming Praxis via Planet Scheme. The following is my solution to the Rail-Fence Cipher problem in Common Lisp [check out the excelente Common Lisp tutorial by Peter Seibel for learning the language].

My solution doesn't follow the solution proposed in Programming Praxis, because, after looking at the proposal, I think there might be a way to express it more mathematical. With this I mean that what we have is a transformation via a discrete wave pattern. That sounded a bit familiar to fourier transformation. If so, if it can be expressed as such, maybe it would be the cleanest way to do it...

So, for now, the solution I present here is based in the straight-forward non-functional algorithm that one gets if analysing the problem as manipulating a string based on a ciclic list of indexes.

Reader Shared items

Followers