Release Notes 3.5.70106

  • [BUG] BSONObj.toChar() Fixed a problem when the string was already escaped.
  • [BUG] The remove command was not clearing the error flag after been executed
  • static link is now separated for OSX and Linux
  • Refactoring in bplusindexp to make it easier to read, also added some checks to ensure page are persisted after the index has been rebalanced
  • [BUG] Fix to an error when loading the namespaces in the webconsole, it was populating the tree wrongly
  • [BUG] Fix to connections that hands when the update generates and error, also added safe thread support for djondbconnection
  • Changes to compile stdc++ statically in Linux this is to avoid problems with Debian
  • Removed the test-dbperformance from the list of tests to execute when doing make test, this to avoid waiting in functional tests.
  • [REL± Bug when the find filter has errors it was generating a null pointer error.
  • [REL] Having 2 io_services were producing unexpected results, rolling back to 1 io_service per thread
  • [REL] BUG: Error in the web console, when the stored value contains \n or \r the JSON becomes unparseable, therefore it was breaking in the browser, to fix this it has been replaced by \\n and \\r to display the returns in the result instead of keeping the \r inside the text.
  • [REL] BUG: Fixed a problem with unescaped characters when converting from BSONObj to JSON char* representation
  • Removed the tools from the CMakeList
  • [REL] Improved the shutdown to delete the PID file when using OSX Daemons
  • [REL] BUG: Fixes to some pointers not properly cleaned on shutdown, those were causing a crash when the server was doing the shutdown process.
  • [REL] BUG: the recreate indexes was lacking of actually creating the index!
  • Fix to the CMakeList as the dqlcommand does not belong to the API any more
  • FEATURE: Implemented support to executeUpdate and executeQuery from the server side to enable native drivers to be easier
  • Boost libraries are now linked as static in all platforms
  • [REL] Bug: Fix a bug that caused host not found in the driver when several connections are opened and closed.
  • [REL] BUG: Fixed an error that caused double freeing inside boost:asio
  • [REL] Implemented a shutdown protocol to persist the indexes and check if the server has been shutdown properly
  • BUG [REL] Special characters like enter or return where causing parser to return errors
  • [REL] Feature: When parsing the dql now it will return the position so user will be able to check what when wrong in that position
  • BUG: [REL] JSON parser was not considering the comma to separate array elements
  • BUG: (REL) Double free DQL Expression when parsing a create Index and it does not parse
  • BUG: [REL] Fixed error that makes the finds to hang
  • Minor: Refreshed driverbase folder and small change in the includes of the bnetworkserver required by linux
  • BUG: [REL] When a connection is disconnected it creates an endless loop in the listener
  • Fixed some issues when stopping with CTRL+C with the new boost implementation
  • Minor fix: the lock in the new Boot Network was not using the lock, therefore the main thread was waiting for the unlocking.
  • Boost asio implementation, the server code has been replaced for asio to avoid reinventing the wheel
  • BUG: Fixed an error with like statments when the % where not used at the beginning
  • Feature: [REL] Support to LIKE operator in expressions
  • BUG: [REL] The DQL parsing was ignoring the extra characters to the right when the expression was valid but with garbarge at the end
  • MINOR: Fix to performance tests, the logger was not compiling under linux
  • BUG: [REL] Minor change in the webconsole, the dbstructure has been collapsed when clicking the namespaces
  • MINOR: Changes to OSX compilation script
  • Lock notify was not included in the new implementation of the webserver
  • Removed unrequired libraries and changed the other of the includes to get rid of the error:
  • Removed a temporal test from the web
  • NULL error when calling the current on an empty cursor
  • Error 35 needs to be handle in OSX as it’s a “busy” code and it will handle the write when the socket is ready.
  • Major: Performance improved the transactions are now indexed so they can check faster if a previous record exists. Removed unnecesary persistent steps in the Indexes generika cialis rezeptfrei.
  • Feature: Implemented Lauch Daemon support in OSX
  • Update DQL was ignoring inner JSON
  • TOP command was not used when executing DQL
  • added the _NOEXCEPT if in linux
  • Updated some exceptions destructors as they are having compatilibity problems
  • Fixes to be able to compile under El Capitan
  • Removed some unrequired dependencies of boost
  • Changes required in the install path for El Capitan OSX
  • Disabled autocomplete in the web console to avoid silly suggestions when typing DQL
  • Error check when updating a record to force revisions to match.
  • Insert in any position in a List
  • Implementation of GreatEqual and LessEqual in query plan
  • BSON Parser exceptions where not exposing the message in some cases
  • fixed a problem when cleaning expressions after a parsing error, the root needs to be set to NULL to avoid double free
  • shutdown Gracefully was not called when shutdown was executed
  • Removed boost dependency in the tests
  • Fix to queryPlanner the equal was only considering Unique indexes
  • ConstantExpression has a conflict betweeen const char* and bool, a new constructor was required to fix this
  • removed the NULL test
  • setLastError didnt have enough space for some errors
  • Error handle executing a tar backup
  • Bug in the name of the txbuffer filename
  • Revision is not automatic based on the date
  • Release Cursor command implemented, the cursors were never removed from the server side
  • Support to NULL when comparing expressions
  • Error handling in commands
  • Bug when there are too many users the OS crashes with “too many files open” message, caused by the txbuffers not getting closed when they are flagged as reusable
  • Changed the constant fo include TOKEN_ this is to avoid a conflict with Windows BOOLEAN macro
  • Merge branch ‘master’ of /Users/Cross/workspace/djondb_-1.34 into queryplan
  • Backup completed, also included in the dql and included the drop namespace
  • Changes initializing DB and Namespace, defaults and NULLs
  • fixed errors with ConstantExpression null and double values
  • FILESEPARATOR has been modified to char to avoid a warning
  • as djondb:string is been used to handle strings in BSON, the bufferedBSON caused a problem when calculating the strlen of the string.
  • Error in comparing and int and a long, it was returning false even that the numbers where the same value.
  • Retusn NULL from conditions was not properly supported
  • Memory Leaks fixes
  • Memory Leaks fixes
  • WebDBAction Capture exceptions
  • Problem with CPU consumed by not clearing up the write_fd
  • Error control in web db actions
  • Removed boost_regex, its not used
  • Web Interface implementation, REST Services: Insert, Update, find, ExecuteDQL implemented
  • Generate dump files on error
  • BSONObj->toChar was having an issue with big BSONs, to solve it was required to use memorystream and move all the components from filesystem to util/io packages
  • Linux/Darwin daemon scripts now implemented. Also djondbd -s will shutdown the server in linux
  • The Buffer Control file was not updated after flushing Buffers this caused inconsistent states of the Buffer elements after restarting the server.
  • The Buffer Control file was not updated after flushing Buffers this caused inconsistent states of the Buffer elements after restarting the server.
  • BPlusIndex versioned
  • Bug: The change to the keycomparator generated a side effect when adding existing items to the LinkedMap
  • Include System Libraries (MSVC…) in the installers
  • Removed the regex tests
  • Removed dependency of boost_regex
  • Removed Antlr dependency when parsing bson.
  • MMapInputOutputStreams Errors log improved.
  • Updated defs.h with the new version 0.3.2
  • Fixed a fatal error when using DBCursors pointing to indexes
  • getLogger in tests fixed to the new implementation
  • Side effect of a change in the signature of the getLogger method.
  • Changes to the getLogger method to make it more robust and encapsulate the constructor properly
  • Error of uninitialized variable, this leads to generate random errors when execting next on empty cursors.
  • Refresh of the includes folder
  • Minor fixes to Linux CMakelist.txt
  • BPlusIndex now receives the maximum size of each element in the index, this allows to do a better use of the space
  • Bug: When the buffer is reused it needs to be reset to avoid carring trash
  • Some problems with the DBStream implementation, detected during the unit tests.
  • Changes in the BPlusIndexP due to a severe bug that caused the logic to create 1 buffer page per index page
  • Rollback the change with Makefile, the make depedencies was not working
  • Fixes to cursors, an error applying the transaction log changes causes to retrieve incorrect number of records, also a change in LinkedMap to improve performance
  • Cleanup of unused files and refresh of HEADERs folders
  • updated php script and README
  • Refresh of the drivers code
  • Bug cleaning the previous cursor during application of tx operations
  • Go Driver implementation
  • 2 Big features created in this commit: Workers and Server-side Cursors

Polyglot Persistent – Tools for the right job

After years of working with IT, as developers we tend to solve problems in similar ways, off course we are humans and we tend to think that the same hammer applies to all kind of problems, but the Internet of Things, where applications are not as straightforward as they used to be, are forcing us to use our brains.

Recently I have been tasked to create an ecommerce application allowing customers to pick from a range of configurations and enable them to provision their services without any other help than the web app, these services include Virtual machines, email, Active directory configuration, etc. Part of the requirement is to enable the platform to grow and be flexible to include new configurations and options based on the customers requirements.

This last requirement requires a system that adapts to changes, like adding new fields, or data that helps the system to achieve the wide range of parameters required for each service and mix of configurations. Also been an ecommerce app, that will charge money to customers, requires consistency and reliability.

How to achieve flexibility in terms of fields without having a deployment problem for every new change? Metadata programming to the rescue, instead of hard coding screens and options we created a set of tables, namespaces as they have been stored in djondb, defining the screen structures, fields, validations, etc. And code an engine that draws everything based on this metadata. This metadata programming is something really hard to archieve using RDBMS, as any new field requires alter tables and update a lot of data, meaning stopping services for new patches, but doing this with djondb is simple and doesn’t require breaking your head, and production at the same time.

On the other hand there’s data structure that is pretty much static and it does not require hard changes over time, things like customer information, orders (who, when, price, etc), etc. Most of these are perfect for a Relational Database, also it will be required to be reported, etc.

Metadata problem

An oversimplified version of the metadata looks like this:


Form: Server Order
Fields: [ RAM, CPU, OS, Data Center, Admin Password, IP Address ]

Off course, each field will have type, required flag, Min length, etc cialis gel. So our simple structure will really look like this:

{
  form: 1,
  description: "Server Order",
  fields: {
      description: { type: "string", caption: "Description", len: 100, required: true },
      RAM: { type: "dropdown", caption: "RAM", list: "ram", required: true },
      CPU: { type: "dropdown", caption: "CPU", list: "cputypes", required: true},
      AdminPassword: { type: "password", caption: "Password", required: true }
  }
}

Now, it’s really simple to create a routine in jquery rendering the above dynamically, and sending the data back to the server in the form of a JSON structure, both structures may grow without changing the code. The code will require changes if a new field type is supported, like “radio”, or others, but both server and client are flexible to render and store the information. Here I’m just showing one of the possible order type, now extend the concept to Email, where you will need to store the user name, password, size, etc. and a full range of other services that users will be able to provision using the platform. On the other hand adding new fields to djondb is not a big problem, just send the new JSON structure and djondb will take care of it.

Processing the order requires a similar approach, you will script some plans based on the above, so using a templating system at the server side that replaces values into script and executes the script is a no brainer. Here using a transactional system over this data makes sense as you need to update the order as soon as the provision is ready, but also store the provision information for later maintenance. No Problem with djondb you have both, you can store atomic multi document transactions and use JSON at the same time.

Relational Data goes to a RDBMS

NoSQL is not really a tool for replacing your full system, it is one of the many tools that you will use to build your application. So we kept relational in a relational database, their infrastructure is based on Microsoft, therefore SQL Server is their preferred choice, information like invoices, customer data, details, etc. where the data structure are almost fixed will live into a RDBMS, linking keys across environments is not really a problem. When dealing with dynamic information that will change over time and requires constant changes will go to djondb, whilst fixed structures are processed in SQL Server.

Another example on where to use the right tool for the right problem is text searching, nor SQL Server or NoSQL are really good for searching in a google style search, for this elastic search helps you to index your texts and provide results that will be linked to djondb and others.

Key decisions to support your architecture

– Fixed data, with no changing structure whatsoever, RDBMS (Pick your choice, SQL Server, Oracle, MySQL, etc)
– Unstructured data requiring transactions (Not too much choice in here, djondb)
– Text search capabilities (Elastic Search)

Now mix all of this in your application and you will use the right tool for the right job, implement your system without any hassle and make your customers happy, using multiple persistence storages into the same application is known as Polyglot persistence.

Hope you like this and let me know your comments, and your approaches to similar problems.

Creating a product catalog with djondb, PHP Restler and Backbone JS Part V – Adding new features

Intro

In our previous post, we created the user interface components of the application cialis frankreich. In this post, we’ll make some changes at database level and we’ll see how our application is impacted due to these changes.

In a relational database world

Now, let’s say that we want to include a new product type in the catalog. Now we want to sell phones too. What should we do in our application to support that?

In a traditional relational database, may be we should create a new table Phone to store its properties such as Manufacturer and Capacity.

But, we don’t want to repeat common properties as Name, Price and Picture, so we better create a Product table to store those properties and make a relation to Book and Phone Tables:

Continue reading

Creating a product catalog with djondb, PHP Restler and Backbone JS Part III – Services layer

Intro

In our previous post, we installed djondb database and used the console to add and retrieve products. In this post, we’ll create the services layer of our application using PHP Restler.

Installing PHP Driver for djondb

Download the djondb PHP driver from djondb downloads page: http://www.djondb.com/downloads.html. In this example we will use the OSX version of djondb’s PHP 5.4 driver. Just open the OSX zip file djondb_phpext_Darwin_i386.zip to extract its content (no matter where).

We’ll see the djonwrapper.php file and the modules folder with the djonwrapper.so file as shown in the following image:

Copy the djonwrapper.so file that is in the modules folder to this folder: /usr/lib/php/extensions/no-debug-non-zts-20100525/ (the folder may change based on your installation) as shown in the image below. if you are not sure where the folder is, you can execute the command php-config –extension-dir and it will show you the folder configured for extensions.

Continue reading

Creating a product catalog with djondb, PHP Restler and Backbone JS Part II – Setting up

Intro

In our previous post, we have explained some aspects of the application we are building with djondb. In this post, we’ll install djondb, the enterprise class NoSQL database and we’ll play around a little bit with the console.

Installing djondb

First of all, download and install djondb. You can get the djondb installer from djondb downloads page: http://www.djondb.com/downloads.html. In this example we will use the OSX version. Just open the OSX package and follow the instructions of the installer.

The installer does not create the database folder, so you will need to do this manually, just run the following command in a terminal window:

    
sudo mkdir /var/djondb
sudo chown ‘id -u’ /var/djondb
    

Sudo allows you to execute admin tasks in your machine, so you can create the folder in the protected area “var”, which is normally used to store the files used by OSX applications.

Once you have the folder create you can run djondb server. Using the terminal type:

    
djondbd -n
INFO:7fff7b008310: djondbd version 3.5.60107 is starting up.
INFO:7fff7b008310: DBController initializing. Data dir: /var/djondb/
INFO:1042f0000: djondb server ready. Port: 1243
INFO:104373000: web interface ready. Port: 8090. Web Folder: /var/webconsole
    

Congratulations, you have djondb running and ready to server your application, the -n allows you to run the server in interactive mode, please don’t close this console session or it will shutdown the database. For any other new terminar related elements open a new session of the terminal.

Continue reading

Creating a product catalog with djondb, PHP Restler and Backbone JS Part I – NoSQL Concepts

Intro

In these series of posts, we’ll create a sample application using djondb database. You’ll see how easy is to create applications using a NoSQL database, especially with djondb, compared to a relational database.

In the first part, we’ll start designing our application and understanding the architecture.

In second part, we’ll install djondb and we’ll use the djondb console.

In third part, we’ll create the REST services that provide the business logic of our sample application.

In fourth part, we’ll create the client application for the final user.

And finally, in fifth part, we’ll add a new feature from database level and see how easy is to make this kind of changes using a NoSQL approach.

So, let’s get started…

Designing the application

The application we’ll create is a simple product catalog. The user will be able to:

  1. Browse available products
  2. See product details
  3. Add a new product to the catalog

We want to use a web browser to use the application, so the client will be an HTML, CSS and JavaScript application created with Twitter Bootstrap and Backbone.js.

Continue reading

Version 3.5 Release

The production ready version 3.5 is now out of the oven.

During the last couple of months I have been working hard to bring the next level of document store, enabling developers to use the power and flexibility of document stores (JSON structures) directly from their favourite dev language to the database, but without losing the power of transactions and consistency.

This new version comes with a new Web Console, easier to access and use than the shell. Just start your server and go to http://localhost:8090 to use it, this supports DQL as explained in the docs and therefore you will feel like in home if you are coming from any SQL Background.

New improvements in the index implementation and look up algorithm provides faster results, the cursors also had a lot of changes.

New drivers on the way, currently the usual PHP, java, Python, C# and C++ drivers are regularly updated but also I am adding drivers on demand, please drop me an email at info@djondb cialis generika kaufen rezeptfrei bestellen.com with your driver request and I will do my best to have it ready within a week.

What about the license? this is still the same, no changes here so you will be able to use it for free, in any commercial, private or open source solution. Do you need a special license? let me know at info@djondb.com and I will be sure to provide you with something that works for your organisation requirements.

In summary I’m very proud of this new version, and I want to hear from you, let me know in which application you are using djondb and I will be happy to add your logo to our customer list, also referencing your web site.

Cross

Version 0.33

A couple weeks ago I released the new version 0.33, this new version included a new implementation of the indexes, server-side cursors to improve speed of results, internal memory managers that speed up the process, workers that allows to control concurrent operations, new Windows Service, new Linux server daemon scripts, etc cialis kaufen schweiz.

I’m really happy with this new version, is working with a very acceptable performance in the projects I’m using it.

The only thing that I stopped doing was working on the social medias, I relay on you to spread the word.

Thanks for supporting djondb.

Cross

What are we working on?

At this moment we are working on the following ideas:

1. Cursors: The idea of returning the full result is causing performance problems, because most of the time the users does not want to use all the results and the time spend to send the whole set to the client is a killer, the first idea was to send the results asynchronous, but at the end the idea of having server side cursors works best, this is almost finished at this moment and hopefully you will see the changes in the following days in the development branch.
2. Installers and compilation: The change to use CMake instead of autotools on linux/mac and Visual Studio on Windows is working really nice, but the change required to remove the compilation of some drivers from the common scripts, now we are working to include all of these drivers compilation on the cmake files.
3. Service Script: A service script to boot up the server using the normal services on linux.
4. GoLang Driver: A new driver for GoLang

If you have any suggestions please put them on the github project and we’ll enqueue them based on priority and complexity.

Cross