Wednesday, October 6, 2010

Organon introduction: Ripple effect (Part 3)

Software modularity and layering became the heart of modern programming and the main reason for large systems to be exist. Unfortunately, no joy without pain. modularity introduce the dependency problem which became a real pain for maintaining large systems which has hierarchical dependencies design, A change in one lower module usually forces a ripple effect of changes in other higher modules.

Database and information stores are usually the lower module in any information system, the problem appears with time and as the development grows changing a decision in the physical model will become more complex and sometimes unavoidable because it will require reflect the changes in all the data manipulation in the system. I think every developer said "O-M-G I have to change tons of queries because of that change" at least once. But why? Usually, a developer creates a conceptual model represents the real world and transform it to physical model which the others modules will depend on it to manipulate the required information, and therefore, data manipulation is strongly depend on the physical model which cause all this problems.

Lets take an example "Who is working in the Blah project?" to answer the question we will need to write a query with full knowledge of the physical model and will access objects not exist in the conceptual design level imposed by the physical representation model, for example (project, employee and assignment) on the other hand, in the conceptual level we will only need knowledge of two objects Project and Employee to answer the question. The problem appears when the innocent customer comes and tell you "oooh, please we start working in groups. can we assign a group to project?"

In this case you have 3 choices after study the butterfly effect of the required change:
  1. Your lucky because you knew the customer will ask for this and you put it in the physical model from the beginning. (God knowledge)
  2. Unlucky you, spend extra hours fixing it. (Man power)
  3. Tell the customer, sorry not everything possible!

Well, I think nobody safe from reach the third option unless you use ORGANON. Not because it will give you the God knowledge or unlimited man power but it will make you use the conceptual model direct without the need to access the physical model in a modular way.

Lets take the above example and see how ORGANON will save most of your data manipulation:
  1. Before customer request the model will contains: Person, Employee, Project and Assignment
  2. After customer request the model will contains: Person, Employee, Groups, Project and Assignment

In both cases the model dose not look different that a normal physical model but lets see how we can query it.
SELECT employee.name FROM employee WHERE project.name = 'Blah'
The great news that the query applies in the first model exactly as the second model! Yes it dose. sure you will still need fix the data I/O like forms to reflect the new data model after all. but imagine with more complex data model how many reports you can make only by knowing which objects you working on.

Also, ORGANON provides modular functionality to perceive the data model for example:

Subsets: You can subset any object in the data model by naming it and reuse it, like LocalEmployee IS A Employee WHERE (type='local') later on you can use it as another object like (SELECT employee.name FROM LocalEmployee WHERE project.name = 'Blah')

Functions: You can define piece of calculations and apply it to a set, like TotalSalaries "employee.salary.sum()" and you can use it with any set related to the function scope like (SELECT TotalSalaries() FROM project WHERE project.name = 'Blah') or (SELECT TotalSalaries() FROM organization WHERE organization.name = 'Accorpa')

This query language called PQL (Perception Query Language) is a extension of SQL and very similar to it, We did our best to make it looks familiar to the SQL developer.

HAPPY ORGANON!

Friday, September 3, 2010

Organon introduction: Information (Part 2)

I love information upon all subjects that come in my way, and especially upon those that are most important. -- Alciphron
Most of enterprise systems deal with information modeling, manipulating, validating and flowing. Usually, one or more techniques are implemented to provide application level access to the information model such as JDBC, JPA, DAO. Unfortunately, available techniques doesn't solve all the problems such as portability, data format, information validity and semantic accessibility. That's why we decide to design a new platform as part of our effort to solve long standing problems.

ORGANON is platform designed from scratch to solve current data modeling challenges and problems and provide set of features:
  • Support many semantic language like RDFS, OWL, OML or annotated classes.
  • Support abstract and final concepts.
  • Support subsets inheritance.
  • Support subset final concepts, for example Age is subset of Integer which will inherit the Integer symbol.
  • Support multiple-parent inheritance.
  • Support scoped declaration and nested concept declaration (not fully implemented yet).
  • Support constrained relations.
  • Automated verification and validation including enumerated types, subset, inheritance, relations and cardinality constraints.
  • Reusable routine, for example Date concept has routine Age returns Integer which can be reused (SELECT dateOfBirth.age() from persons)
  • Defining objects in time, for example referencing task.assignee to employee in "present" where 'present' constraint by status is active.
  • Complete metadata query and reasoning API.
  • Complete Event/Subscription API.
  • Schema compiler and validator.
  • Source code generation from model to java classes.
  • Predefined queries or percepts.
  • Strongly typed symbols.
  • Transparent persistence and Memory API.
  • Multiple I/O formats including XML, JSON, RDF.
  • Rich query language including support for query abstract concepts, for example (SELECT * FROM OnlineUsers WHERE Weather.status = 'Fine' AND Employee.company= 'Accorpa')
  • Rich instances inferencing API.
  • Global annotation reasoning API.
  • Binding API similar to JAXB.
  • RDBMS store implementation (JDBC)
  • XML database store implementation (not fully implemented yet).
  • J2EE architecture Integration (not fully implemented yet).
  • Maven integration.
ORGANON (codename: LIBERTY) focus on the information accessibility and transformation between different data representation formats, the below diagram shows the information flow (Symbols are strongly type objects)

We started writing in ORGANON since two years ago. it reaches 120K lines of code (coverage). We plan to move from alpha to beta in about month and still have some issues with licensing.

120K lines have a lot of things there to talk about

HAPPY ORGANON!

Monday, April 12, 2010

Organon introduction: Data model (Part 1)

The data model is core component of any information system, Data model describes how data are represented and accessed by defining data elements and relationships among data elements. Most of modern system use OO language as front end to reflect data model stored in back end store like RDBMS. As history shows the development of different techniques to store different data model approaches to persistence store like RDBMS, beginning with applying any pattern to abstract the database access layer from objects layer and ending with ORM library to manage mapping objects to persistence store like Hibernate or JPA.

Most of existing ORM libraries focus on mapping OO classes to RDBMS database and provide set of flexible configuration/API to give the developer more control on the mapping in order to speedup development and deployment of the application, also, some of those libraries provide extension query language to eliminate the redundant declaration of the relation between objects like OQL, HQL and JQL.

Now you can easy declare class Person and transparently persist it and query the available persons, but the question still alive, is that all i can do? the answer properly is No. We still far away from the Yes answer, but why:

  • The person data model is not cross platform/language, you can't use it out side your system.
  • The logic inside the person data model is known only by the class itself.
  • Some of person relations is not descriptive for example, Person has age (Integer) represented in native datatype or immutable object, there is no enough information for the system to answer some question, so the answer of query "Give me all objects where their age more than 3" would be "Excuse me, What is the age?"
  • There is no inheritance between subsets (Boolean expressions), for example, Man is subset of Person where gender equals 'Male', you should be able to extend Man as a class.
  • The relation cardinality is not considered constraints (One to many relationship), do you ever heard about person has 100 child?
  • There is no support for multi-parent inheritance (interface inheritance)
  • In most existing ORM libraries there is no clean API to access and query the metadata of the managed data model.

There is a lot of effort in many projects to solve the above problems you may check out Jena, RDF syntax and OWL. In the next post i will talk about Organon way to solve the above problems.

Sunday, April 11, 2010

Welcome to Pushing forward!

This is my first blog on Pushing forward!

Lets see how far i can manage Pushing forward! is about Organon, ORM, RDF, Semantic Technology and more .. I'll be mainly writing on my own area of interests.

 Cheers