Boredom for Professionals

This text is intended for those readers who are interested and need to know and understand both functionality and performance capability of the proposed platform.

1. General Architecture
2. Data Safety and Reliability
3. Scaling
4. Data Consistency
5. Access Restrictions
6. Development Environment/Design Tools
7. Multiplatforms
8. Other Ready Platform Services

General Architecture

The system is a 3-tire application with the following layout:

Both Oracle 11g and PostgreSQL are supported by the application.

The architecture is built to provide a high level of accessibility and to prevent hardware or software failure. In addition to the high level of security, such scheme has its additional advantages:

• Business logic is not dependent on the client applications;
• Client’s workstations have no direct access to the database server. Thus, the intrusion into the workstation does not provide direct access to the database server.
• Centralized Updates (client applications are updated from the application servers)

Some of the other less obvious advantages will be described below.

Application servers and user application are written in .NET Framework for C# platform. The interaction with application servers takes place through .NET Remoting. There are other client applications, such as:

• Ultima::eStore serves the basis for the website (or websites). Strictly speaking it is a ready platform for an online shop loaded with full list of services. Online communication with the DB, such as reading and recording of information, takes place through web services (SOAP);
• Ultima::touchStore is an online application designed to work on self-service terminals with touch-screen panels. The touch-screen interface is nice, convenient, and has functionalities similar to those of the Ultima::eStore. This application is written in .NET Framework and communicates with application servers through .NET Remoting;
• Ultima::WMS is the firmware platform serving the basis for warehouse control system which is configured to the needs of each individual enterprise;

The system design allows to support different storage schemes — sectional, address, task, serial number record, bar code generation, tasks routing to packaging/storage/handling of assorted ladings, record of weight and size parameters of the product line (linear dimensions, weight, quantity per box, fragility and so on), including required transporting capacity to handle the assorted ladings, automatized record on physical limits of the warehouse (size and weight limits of storage and processing at the warehouse, maintenance boxes, cells, product lines and so on), support of line acceptance/shipment of big and very big batches and continuous processes, etc. The software is written in.NET CompactFramework. The specialized shockproof Wi-Fi notebooks equipped with bar codes scanner are used as the terminals (currently, the main recommended model is Motorola MC3000, and advantageous Motorola PPT8800, which may still be available, but have already been taken out of production.

A set of applications provided within the conventional client-server architecture makes it somewhat difficult to support a coordinated business logic model. Especially serious problems are related with the site that is a fully functional industrial online shop. Such elementary business logic like ‘Get a free gift with order over XXX’ creates serious issues related with the website coordination and with the master application, such as calculation of the total amount of the order, gift availability check and so on. Any designer has encountered with similar examples in practice. In our system, the unified business logic storage and full integrity of the site with the application servers cluster is an ultimate solution for such problems.

A brief description of the workstation access: Logical organizing scheme is shown in the following picture:

Such organizing allows simplifying the network administration. Workstation may consist of a cheap slim client without HDD (it uses flash memory) and even with no processor cooler.

This is to say, there is nothing that can be broken.

All the workstations are remotely administered. In case of very unlikely failure of the workstation, its repair takes about one minute: the workstation is simply replaced by a new one and the new workstation is simply plugged in. No additional set up is required.

This significantly increases the reliability of the infrastructure (provided the servers are used in a high quality datacenter and have reliable reserved commutation channels). The efficiency of system administrators’ work is enhanced at least ten times.

Data Safety and Reliability

The system is designed to work with the databases performing several thousands of operations per second, the size of which is measured in terabytes. Clearly, to loose data in such a situation means winding up the business, as it is virtually impossible to restore the necessary information from hard copies within the reasonable amount of time.

That’s why, in order to keep the data safe, the vulnerable nods are duplicated. For these purposes, our system uses Physical Standby Oracle Database and clusters among application servers. Thus, even the failure of the main server will not cause data loss nor even temporary shutdown of the business, as the system will switch to use the reserve server and will continue its normal operation. Similarly, failure of any application server will not affect the functionality of the system, as the clients’ applications will switch to work of other application servers. This is also true about the websites. Besides:

• The platform is designed in such a way that it does not store the data in clients’ applications (with the exception of Ultima::eStore in which the data stored are limited by minimum of required information that is necessary to display the functional on-line catalogue — and which is, by definition, commonly known information);
• The site communicates with the application servers through web-services only and has no access to the central DBMS. Thus, even in case of a successful hacker attack the intruders will not obtain access to the ERP database;
• The use of slim workstations allows to exclude the possibility of storage and divulgation of the confidential data by the employees;
• Communication level supports zero knowledge proof authentication (no passwords or password hash codes are transferred through the network), symmetric encryption of the data with the most modern algorithms (AES, 3DES and so on), and transparent data compression in the dataflow. Encryption and compression settings may be adjusted for each individual user or application; such flexibility allows balancing the additional load on the CPU with the need to compress/encrypt the users data and connecting through own unprotected networks and/or without broadband internet access.

Scaling

Practice proved, that this system provided admissible performance level under the load that was generated by 1,500-2,000 users working simultaneously (and >70,000 website visitors). Of course, such load was not served by a cluster of application servers and not by one PC.

Application servers are able to form cluster and exchange the necessary data. Alongside with that, the application distributes the load among the database servers and uses reserved DBMS server to generate reports and to process certain requests.

The scheme itself is a mixed OLAP-OLTP scheme with several spider formations. Thus, the system does not have excessive blocking power and on the other side allows the receipt of real-time reports. System scaling is easily done by either increasing disk surfaces on the main database server, or by adding report generating reserve servers or by adding new application servers to the cluster.

The insiders know that the dead lock resolution is very resource intensive. We fitted the nucleus with the system that sorts all the operations related to data modifications by internal codes, which excludes potential intercept of the changed data by two different transactions, and thus reduces potential dead lock cases.

Data Consistency

Data consistency is understood both as consistency of references and logical correctness of the data. Built-in DBMS is responsible for consistency of references, while system nucleus controls logical correctness. Logical data consistency is provided by means of strict control over the two following principles:

A. Double Recording controlled by the system nucleus at any time when material assets undergo any operation. It is impossible to get around that principle and to change data without having access to the Oracle tables. Besides, consecutive double recording allowed implementing such functionality feature as FIFO fair cost value calculation; such calculation is done within reasonable time and using useful data capacity.

B. Transaction Consistency on the Level of Application Server. Any transactions performed on the application server, starting from the so-called ‘log-in’ or activation of any function from the external application, and to the ‘exit’ or to the return from that function, will either be completed successfully, or all the transactions will be aborted.

The designer will not need to consider the source of activation of the code, and the user will be assured that his/her transaction will be either completed in full, or will not cause any changes at all. For example, the user saves the receipt for the goods arriving at the warehouse. If the order contains ‘special order’ items, the clients order will have to contain such items. If any of the documents will contain exception (for instance, if the client does not have sufficient balance and this document may not be changed), the entire chain of the operations will be cancelled. Neither clients’ orders, nor receipt for the goods will be changed; the system will save in consistent mode.

Access Restrictions

Each company needs tools to determine users’ access to the data. Ultima ERP provides two complementary schemes of access control:

• Branching Scheme(or the internal term: role scheme) is a generalization of the conventional idea of access rights. The roles may be called rights, and in many cases they coincide. However, the use of roles allows creating more complex versions of limiting access. Only the imagination of the business processes architect may create limits thereto. The system helps the system administrator by providing specific tools: merging several roles into one, permitting evident exclusion of the role (semantics: �to warehouse rights without inventory rights�), searching users with given roles, and so on. The main idea of the �branching� consists in performing a certain �branch� of the code in dependence on the existence or absence of the role. For example, a role may permit shipping the goods to the client with outstanding debt. It is clear that to program such a right we need a corresponding code. Roles are not always rights, as both branches may perform meaningful code, for example different users may be shown different interface of data input, depending on their role.
• Predicate Device or limit on line levelThe predicate access scheme allow setting up so-called predicates [arbitrary set transformed into (0, 1) set or, in other words, return function always returns to 0 or 1]. Such schemes allow the administrator to form as many complex limitations as needed; for example, to see only those users who were created by the current user. Or to see the counteragents created by colleagues and to edit only the counteragents created by the user him/herself. It can be various.

Development Environment/Design Tools

The objects of business logic, reports, events processing and display and print forms constitute the so-called configuration of the system. In its turn, each configuration may have its own history of versions. The application server works with one configuration version at a time. The schemes of creating different configurations versions support all basic features of versioning systems, such as Fork, Merge, Diff, etc. Tagging configuration versions is supported. Such scheme allows each designer to work in his/her version of the configuration, and then to transport the changes into the version that will be used by the test server. As a rule, the system runs 3 versions simultaneously:

• Production;
• Test;
• Development

that are labeled accordingly. When working in a big team, it is convenient that each designer has his/her own tag to label the version he/she is working on. A separate tag may be created for the version correcting ‘hot’ bugs. The server has two operation modes: static and dynamic. First mode is used for Production and Test server and does not allow modifying the structure of the system of business objects, or introducing any changes to the configuration, while the server maintains high performance. The dynamic mode is used mostly by designers and allows changing the scheme configuration ‘on-the-go’.

Business rules of generating objects, reports and so on are done in Visual Basic.NET, and the designer has the latest language resources: LINQ (to request and obtain business objects), generics, lambda-functions etc. For less standard tasks, the server supports MEF (Managed Extensible FrameWork) and publishing services (Singletone abd SingleCall) via .NET Remoting. The platform provides services for LINQ requests from clients’ applications to obtain access to the system objects, while the real work is done on the application server with all the rights of access duly checked.

Designers’ environment allows viewing the chart of configuration versions, seeing and editing methodical data (description of system business objects), editing scenarios and creating their own services to store business logic. If the server operates in dynamic mode, the changes will be accessible for the use immediately.

Any object of the system may be labeled by a tag, which simplifies the transfer of functions between different versions of the system (or even separate configurations of the system). Thus, having labeled certain scenarios and objects of the system with the tag ‘complaints and suggestions’, you may obtain the list of complaints and suggestions for further viewing or export.To debug an application, it is possible to activate routing of the system scenarios log-ins, roles checkout, predicate checkout, referral to constants and SQL request in process that contain stack of log-ins and profile information.

The nucleus of the system supports business logic UNIT-test frameworks, while the nucleus itself is completely covered with UNIT-tests, which guarantees its stable operation.

A convenient codes editor is provided to program system behavior; it supports syntax highlight and IntelliSense (including for system objects), trace routing and debugging methods, multi-purpose log-in (also in EventLog) and so on.

Forms generator is provided to develop clients’ applications (intranet) by using metadata with an option to change layout form. In most cases this enables to reduce the development to declaratory description of the system object. In more complex cases requiring more complex behavior of the form, Visual Studio or ready set of data display components may be used.

The set includes TextBox, DropDown, Label, Radio, Checkbox, Grid and about 50 other components. The designer is free to use any third-party components.

The system provides ready tools to generate complex parametric reports with various filters and amount of details. The designer only needs to input the so-called ‘basic’ request (OLAP-analogue: cube) and to specify the number of details and the filters to be used. The system will generate the form of the report. Moreover, if the basic request is based on the table of facts, it is enough to describe the table of facts in terms of Measurements, Variables and the system will build all the necessary filters. The knowledge of the structure of the data allows the system to define the request with ‘unlimited details’. For example, the system contains the description of the total: ‘Leftovers in the warehouse’ as the cube with the measurements ‘warehouse’ (from the warehouses list’ and ‘goods’ (goods list), and with the ‘quantity’ variable. The warehouse contains indication of the office, and the office contains indication of the city. The goods contain indication of the producer, and the producer contains indication of the country. In this case the system allows choosing the following chain of details: warehouse.office.city and goods.producer.country.

Any other properties of linked lists may be chosen at analyst’s discretion. Besides, raw (non-formatted) data may be transferred for further processing into a summary table (whose features are similar to those of Excel), or into special OLAP analytic interface allowing to build charts, make profiles and perform other standard analytic functions.

As it has been shown above, many problems are solved through declarative statements, and the system creates user forms based on the described metadata, which significantly simplifies the design and support of configuration.

The platform provides tools to develop print forms; the data required to generate print form are generated by the person processing the data. This allows creating print forms of unlimited complexity. Export to PDF, Word, Excel, TIFF, HTML is available.

In addition to the access schemes through LINQ or through other application server services, the designer may perform random SQL request. If the designer does not evidently modify the level of access, even the random SQL requests will have all the required access limitations applied. The result of the request may be obtained in detached-data model (for example as a simple DataTable), or in attached model, which sometimes is required to process the very large results (million lines, for example). Besides, if the system contains any read-only standby server, and the current request does not require access to temporary tables, the designer may want to use the connection with one of the standby servers.

To set up the scheduled tasks launch the system provides corresponding interface. Those who worked in Unix-systems know the method of defining the graphics, which is similar to CRON. The traffic distribution may be done by linking certain tasks to preset application servers.

The system supports multilingual user interface. The names of the objects properties that are displayed in forms and tables in the design environment may be translated into other languages, while the client application will display the user interface according to the user settings. Similarly, if any of the scenarios of the system requires to display the message (such as: ‘week password’), a set of resources allows to display the message in the required language.

Multiplatforms

The application server is developed to work in .NET FrameWork in any supporting Windows, and in Mono platform. This allows launching the application servers in Linux servers. In addition, the application server (together with DBMS) may be unwound in Amazon Cloud. Support of clients’ applications in Mono platform is included in long-range planning, and only depends on corresponding development of the Mono platform itself.

Two most convenient frameworks for Unit-testing: NUnit and VSTest are supported (which is convenient for Visual Studio designers). The tests classes are invariant in relation to the platform.

Other Ready Platform Services

• Call-center. The system allows integration with two IP servers: Asterisk and FrontRange. In both cases the platform supports two-way connection with IP server, i.e. allows transfer of commands (dial number, answer the call in-queue and son on), as well as to act in according with the events (incoming call, create queue and so on). General logic scheme of connecting to IP server is shown on the picture below:
  
  Asterisk is a free open-source server with low cost ownership, which will satisfy the needs of any company with relatively low calls flow (roughly up to 5,000 calls per day and up to 500 calls per hour). Starting from 10,000 per day or at the peak of 1,000 calls per hour, we recommend using commercial and more reliable when working with high traffic FrontRange.
• Ultima::WMS. This platform provides API for:
  
  • — Bar-code scanner control (turn the scanner on, receive scanned bar-code, and so on);
  • — Device control (sound when scanning ba-code or in case of error);
  • — User authorization and access rights control;
  • — Receipt of the data from application server;
  • — User interface control;
  • — Automatic update of the application;
  • — Communication with the application server;
  • — Ready classes and methods of generating data tables, documents and lists;
  • — Display editable tables, menus and other control tools of the user interface.
  In addition, Ultima::WMS hides the details of data transfer between the client’s application and the application server, which enables the designer to use screen logic only, while the platform performs detailed serialization and objects transfer. The user interface may be set up similarly to the master client’s application. A special server may be dedicated to process temporary logic layer of interaction with data collection terminals in the cluster. The dedicated server is needed to perform complex warehouse logics and may cash data, thus reducing the DBMS load and send messages about data changes to the connected workstations.
• Sending SMS via GSM terminal or SMPP port with possible limitation of the sending time at night;
• Email support, including HTML emails with attachments;
• Messaging to users systems. The messages may include reference to the system object that will be available (the message will have a special key);
• History logs of any object modifications (user, data, object). It is possible to find out the author of changes of any system object at any time;
• Print server allowing identification of the printers within the system and organizing automatic printing, and detailed print register. The use of this server will reduce the traffic created when the offices are located in different places: the print server only receives printer identification, print form and necessary data. The rendered print form may not be transferred via network.
• Access to other DBMS. The platform provides methods of accessing MySQL