Tag Archives: PostgreSQL

A new North Star has risen


While this post is not going to be about pulsars, black holes or any other form of astrological phenomenon…It still is going to cover one of the most exciting and fundamental shifts in the IT industry today.

A tale of two convergences

This tale is discussing convergence.
To understand the significance of any convergence, it is necessary to understand the lines that are coming together. Please indulge me when I review these.
The challenging bit is the fact that inside both of the lines there are only the very early glimpses of this new era. They are still very much busy with the day-to-day operations in open source or conquering more of their base realm.

Why Postgres is the answer

I migrated myself from Oracle to Postgres! Moving from a steady path as an Oracle ACE to this—at least for me at that time—brand new world of open source data management. If you want to know how and what, I wrote a trilogy about that here, here and here.
After working in the Oracle realm for almost a quarter of a century, moving from Oracle to Postgres truly felt like following a new North Star. It has proven to provide good guidance.
With the phrase still in my head—”horses for courses”—the PostgreSQL Global Development Group does focus exclusively on Postgres. Coming from Oracle, though, it made me wonder who focuses on all the other aspects that vendors create to make their systems work. This “emergence from the red bubble” made me realize that there are much broader challenges, which leads to the “second thread” for convergence, in this story.

I like to think this puts me in a position that allows me to oversee this part of the spectrum (databases and data management) to a certain degree.

Note: if you realize where the data management industry is moving, Postgres is the answer.
Hold that thought!

Brain-breaking challenges

Over the years, much focus has been on infrastructure. Simply because it is expensive, tedious, error-prone and has lots and lots of room for improvement.

  • Infrastructure as code – your server is not your pet, you do not tend to it, you replace it.
  • Cloud infrastructure – your server?? It is not your server… you just use it, you are server-less.
  • Many more of these developments have shaken the world since I started in IT.

There are a number of drivers that we can distinguish and that play a part here:

  • Cost needs to go down!! The ever emerging CIO challenge is: do more with less. And, we are succeeding in that, year after year…
  • Speed needs to go up!! We need more features for our applications, we need them sooner, we need them working more flawlessly.

Working in IT operations during various points in time, these elements have always given me the hardest brain-breaking challenges.

The winner is…Postgres

It is really, and undeniably so (or at least it is so in my book), Postgres has won, and where it is still competing, it will win <full stop>

Why?

There are several reasons why I think—why I know—this is going to happen.

  1. Publicly governed open source, community driven open source, give it a name. It is not a for-profit thing that is behind the technology…Hence, there are no barriers or boundaries to opportunities and direction.
  2. Relational will never die. Dr. Michael Stonebraker said it himself, there is going to be no post-relational era, and the more data management / processing methodologies we get, the more we will need SQL to make sense of things.
  3. Metcalfe and Reed’s laws. More ideas, more contributors, more firepower to Postgres. Postgres’ strong and unwavering foundation grows and evolves. The foundations laid by the PostgreSQL core team will continue to feed the fire of Postgres for the foreseeable future.
  4. Datawarehouse, Datalakehouse, Graph, NoSQL, NewSQL, distributed, and what other mumbo-magic words you can put together. We’ve seen it in the past and we will see it again…it will all converge back to Postgres. It’s inevitable. The latest  evidence: Apache AGE. I rest my case.

Stop for a minute… just a brief pause to think about some of the implications of this…Put prejudice aside, and just consider this: the Power of Postgres, the most transformative tech since Linux.

Kubernetes (K8s)

Enter Kubernetes! If you have not yet done so, I strongly recommend watching the two-part documentary on the origins of K8s here and here.

Once upon a time, at a conference somewhere in Eastern Europe, a keynote speaker was talking about IT operations. At some point in the story, he told us that he was on the phone with his CEO after a failed deployment. He said; “Look, Sir, the application worked on my computer!” and his CEO replied: “Well, that’s all good, except that I am not paying you to make it work on your computer, but I pay you to make it work on my computer”.
In my opinion, this is one of the key elements that container infrastructure solves. Immutable infrastructure, as part of running Postgres, the Kubernetes way. A critical new way of looking at integrating Postgres as a core element for data processing in a true cloud native manner.

Apart from doing infrastructure very well, K8s turns the world of application development on its head. Monolithic Applications become Microservices based solutions, the paradigm that allows for things such as Continuous Integration / Continuous Delivery, and many more of the super-cool things that DORA describes.

Oh, and did I mention that Kubernetes is also a publicly governed open source community driven project?! Check out what the CNCF is all about! I won’t go on about Seven of Nine this time, I promise.

Data on Kubernetes

So basically, there you have it!

We have seen AMAZING things from Postgres!
But they never really focussed on deployment and alike; “horses for courses”.

We have seen AMAZING things from Kubernetes!
But they never really focussed on data and alike because developers need to build features!

Under the awe-inspiring guidance of the CNCF, we have an actual Data on Kubernetes Community!
A first, profound and fundamental step on the path of convergence, where Postgres meets Kubernetes and we start enabling a new era that might start bringing a couple of answers to some age-old (well, as far back as the invention of computers, really) and some newer challenges such as:

From here it is basically: “Hi-ho, Silver! Away!”
There is no stopping this, or as a colleague once paraphrased Babylon 5: “The avalanche has already started. It is too late for the pebbles to vote.”

This is the new North Star.

In the end, we’re just getting started

  1. Oracle to Postgres—well, that’s done. What has not yet been migrated will probably die out at some point.
  2. Postgres is established, no debate there.
  3. Kubernetes is so strong, so appealing, it answers so many questions that it will be with us for quite some time.

There is, though, this one fundamental gap. However you twist or turn it, the user of your app needs data, otherwise what’s the use of all of your mega cool features?

Postgres and Kubernetes, the two most powerful technologies of today answer that question.


Postgres in the Enterprise, Real World Reasons for adoption

The lead-in

Friend and colleague Bruce Momjian already shared in his presentation “Will Postgres live forever?” (https://attendee.gotowebinar.com/register/8793191803818201345) an overview of reasons why companies adopt open source software (OSS) in general. This overview was based on a survey done by Black Duck Software in 2016 (https://www.slideshare.net/mobile/blackducksoftware/2016-future-of-open-source-survey-results). The survey ranks databases on the second place of technologies that companies are moving towards OSS in.

In this series of short blogpost I want to highlight some of the practical cases of Postgres adoption in the Enterprise from our day-to-day practice, and see which of the general factors of OSS adoption we see with Postgres .

IBM DB2 to Postgres

The first of the cases deals with a semi-government organization that currently has an application landscape based on IBM DB2, Microsoft SQL-Server and Oracle. Our primary engagement is to support their move of their application landscape from IBM DB2 to Postgres.

The initial driving force behind this decision was the staggering renewal-cost this organization was facing for their DB2 licenses. These licenses included IBM’s Deep Compression option (https://www.ibm.com/developerworks/data/library/techarticle/dm-1205db210compression/index.html). This option reduces disk-space by as much as 50%. More importantly it reduces database IOPS to meet the performance of the applications. Moving from DB2 on Z/OS to DB2 on Linux, the company faced a performance deficit which they initially resolved by using this Deep Compression option.

This option was also a grounds for worry. Obviously an expensive database option, that was known to bring real benefits to DB2 database performance, would outperform Postgres, right? Would we be able to come near DB2 database performance when running on Postgres?
Discussion with our own Postgres experts could not take away these concerns. We would have to test and validate.

The project had already kicked into gear when EnterpriseDB was engaged. Some of the software had already been migrated and converted to fit on EDB Postgres Advanced Server (EPAS). For much of the data and schema migration, the software by Spectral Core, named Full Convert (https://www.spectralcore.com/fullconvert/).

The outcome

Part of the extensive test-program of the migration project was performance testing, which also included a simulated full pressure practice test with load-generators.

The surprise was then also quite absolute, when the final results of the performance tests were released. EPAS was capable to keep up quite nicely with the original DB2-setup, and even improved performance in some specific tests.

Performance tests concluded, June 28 2018

Test Number Postgres Runtime
% of original DB2 runtime
Number 1 87.2 %
Number 2 101.0 %
Number 3 109.4 %
Number 4 79.3 %
Number 5 70.0 %
Number 6 109.5 %
Number 7 106.4 %

In analysis of this particular project, we can conclude that the reasons for this project to choose Postgres are:

  • Primarily #5, Cost reduction. Database running cost on an annual basis can be reduced in access of 80%
  • Secondly #2, Freedom from vendor lock-in. Postgres is a community open source solution having no single company controlling the solution.

More details

I am curious to learn what details you would want to read here in order to help you assess your specific situation! Let me know in the comments.

Swiss pgday 2018

The cool thing about zooming out… is that your world appears to get bigger.
Being personally now no longer bound to Oracle, having the opportunity to work with PostgreSQL, also gives the opportunity to go new places and explore new possibilities. One of the cooler things is: participating in Postgres conferences.

Conference vibe

Where Oracle conferences, although having some deep technical aspects, tend to lean towards the business aspects of technology, especially today with Cloud first / Cloud only… PostgreSQL conferences tend to lean towards engineering. What things are we – that same Postgres community – building in and around Postgres. What do we think about these developments and how can we improve them.
Postgres tends to have anywhere from 5 to 10 different directions in which the product is being developed. Lots of people check, test, improve, criticize and comment on all these developments.
A significant difference, somehow.
The atmosphere at PostgreSQL conferences, though is also simply super cool. New people to meet, new ways to incorporate.

Swiss pgday

I had the opportunity to join and participate in the Swiss pgday (find the program here) in the beautiful town of Rapperswil, at the university of applied science (HSR). Together with my friend and colleague, Postgres founding core team member, Mr. Bruce Momjian, I joined the event.
The Swiss Postgres community booked a nice result with a 30% higher number of participants. In two tracks, over 12 talks were delivered by local and international speakers on many aspects of Postgres, from a more business perspective on Postgres to the new things that come with Postgres 11, and can now be tested by anyone who wants to!

With all these shifting panels, with this second wave of Open Source rolling, that is now happening… More intricate systems, like relational database management systems, are now being offered and adopted.
It makes sense to zoom out as the opportunities increase so rapidly and in ways never foreseen.

I challenge and invite you all; come on board and ride this wave with us.

Why I picked Postgres over Oracle, part III

This is the final episode of this short series of blog posts on some of my drivers for moving to Postgres from Oracle.
Please do read Part I and Part II of the series if you have not done so. It discussed the topics “History”, “More recently”, “The switch to Postgres”, “Realization”, “Pricing”, “Support” and “Extensibility”.

In summary:

  • Part one focused on “why not Oracle anymore, so much”
  • Part two discussed on the comparison between PostgreSQL and Oracle
  • Part three talks some more on what Postgres then actually is

Community

One of the more important things to be really, really aware of is that Postgres is “not just open source“. Postgres is “community open source“.

Now, why would that be important, you might wonder.

We all know what open source stands for. There are many advantages to an open source system, and in our case, an open open source database.
A number of arguments are in this blog post series. If you take this one step further though, and realize that Postgres is a community open source project, what are  extra advantages?

A community open source project is not limited, in any way, to any one specific group of developers (let’s call them a company). For example, let’s look at MongoDB. This is an open source database, but it is developed by MongoDB inc.
It is, in essence, controlled by MongoDB.

Postgres is developed by the Postgres Developer Community coached by the Postgres Core Team.
This makes Postgres incredibly open, independent and it enables its developers to truly focus on actual business problems that need to be solved. There is no ulterior drive to satisfy commercial goals or meet any non-essential requirements.

Development

A very important discriminator, that only became this clearly and apparent to me, after I dove into Postgres some more, is the development…

The actual development of the database core-software is done by this community, we’ve just identified.

“Well, yes…” you might say, this is what open source stands for. But the impact of this extends well beyond support, as I mentioned in part 2 of this series. The ability to be part of where Postgres goes, to have actual influence on the development, is awesome, especially for a database platform in the current “world in flux”.
Postgres users don’t necessarily have to wait until “some company” decides to put something on the road-map or develop it at their discretion. These company-decisions will mostly be driven by the most viable commercial opportunity, not necessarily the most urgent technical need.

The development of Postgres is more focused on “getting it right”.
One nice example is the Postgres query optimizer. The Postgres community hates bugs. When bugs start to get discussed, it results in many emails within the community, which stand for a lot of reading!
Many bugs are fixed very quickly, so that this email storm stops!
For optimizer bugs therefore, turn-around times (from reporting to having a production-fix) can be as low as 72 hours, so even for mechanisms as complex as a query optimizer.

Invitation

I would like to invite all of you in the Oracle community to take a look at the Postgres query optimizer and share your concerns, worries, bugs or praise with the Postgres community.
If you want to, you can share this with us at the https://www.postgresql.org/list/pgsql-hackers/ email list. We are looking forward to your contributions!

Future

Oracle

I can only speak from what I see. What I see is that Oracle is becoming an on-line services company. I see them moving away from core technology like the database and accompanying functions. Oracle is more an more and moving to highly specialized applications aimed at very big companies.
Chat-bots, social media interactions, integrated services and more, delivered from a tightly integrated but also tightly locked set of Oracle owned and operated data-centers, or rather, the Oracle cloud.

Is this useful? Of course, there will be targeted customers of Oracle who will continue to find this all extremely useful, and it will be, to them.
It this for me? No, not really.

Postgresql

In the beginning, Linux was not something anyone wanted for anything serious. I mean, who wanted to run anything mission critical on anything else than Solaris, HP-UX, VMS, IBM? No-one…
And that was just a few years ago. Imagine!
Today in any old data-center, if you would eliminate the Linux based servers, you would not have much left.
This same thing is now also happening in, what I guess is the second wave of open source. More complex engines are being replaced by open source and the ever present relational database engine is one of those.

Why? Price, extensibility, flexibility, focus, you name it. We have seen it before and we will see it again.

EnterpriseDB

If you permit me just these few words.

I think EnterpriseDB is extremely important for PostgreSQL. We have been fighting on the forefront since the beginning, supporting PostgreSQL’s move to the Enterprise. EnterpriseDB has been and will continue to spend a large amount of our resources to PostgreSQL. We are a PostgreSQL support company. We just have been not very good at patting ourselves on the back…
As a company we are doing extremely well, simply because Postgres is rock solid in all facets and ready to take on the word, even the most daunting tasks – and beyond.
This will continue as Postgres will continue in this second wave of Open Source.

I thank you for your attention.
If you have addition questions or comments, please do not hesitate to contact me.

Why I picked Postgres over Oracle, part II

Continuing this short series of blog posts on some of my drivers for moving to Postgres from Oracle.
Please do read Part I of the series if you have not done so. It discussed the topics “History”, “More recently” and “The switch to Postgres”.

Realization

In the last months, discussing Postgres with my Oracle peers, playing with the software and the tooling, I actually quite quickly realized Postgres is a lot cooler, at least to me. Not so much of the overly complicated technology, but rather built to be super KISS. The elegance of simplicity and it still gets the job done.
Postgres handles a lot the more complex workloads than many (outsiders) might think. Some pretty serious mission-critical workloads are handled by Postgres today. Well, basically, it has been doing this for many, many years. This obviously is very little known, because who would want to spend good money on marketing  for Open Source Software, right? You just spent your time building the stuff, let somebody else take care of that.
Well… we at EnterpriseDB do just those things, …too!

And, please, make no mistake, Postgres is everywhere, from your fridge and video camera, through TV set-top boxes up to major on-line banking software. Many other places you would not expect a database to (be able to) run. Postgres is installed in places that never get touched again. Because of the stability and the low to no-touch administrative character of Postgres, it is ideally suited for these specific implementations. Structured on some of the oldest design principles around Postgres, it doesn’t have to be easy to create the database engine, as long as it “just works” in the end.
Many years ago, an Oracle sales director also included such an overview in his pitch. All the places Oracle touches everybody’s lives, every day. This is no different for Postgres, it is just not pitched anywhere, by anyone, as much.

I have the fortunate opportunity to work closely together with (for instance) Bruce Momjian (PostgreSQL core team founding member and EnterpriseDB colleague). I also had the opportunity to learn from him some of the core principles on which Postgres was designed and built. This is fundamentally different from many other software projects I know and I feel it truly answers some of the core-requirements of database projects out there today! There is no real overview of these principles, so that’s on my to-do list.

Working with PostgreSQL

Pricing

Postgres is open source… it is true open source. It is even a true community open source project, but more about that later in the next installment.

Open source software is free to use, it does not cost nothing!

But, wait! Open source does not mean for free! How…, why…, what do you mean??

Well… you need support, right!?
The community can and will help you, answer questions, solve some of your problems. But they will not come in to install, configure and run Postgres for you. You will need to select and integrate your specific selection out of the wealth of tools. You basically have a whole bunch of additional tasks to complete to get your Postgres platform sorted out.
Companies like EnterpriseDB can help you mitigate these tasks. This allows you to focus on the things you actually want to achieve, using Postgres

In comparison to traditional database vendors, the overall price of your solution will absolutely significantly reduce when using Postgres as your open source database engine.

Support

A significant difference between Oracle (for instance) and Open Source support services is interchangeability.
In the end, Oracle support can only be given by Oracle. They are the only ones that have access to the software sources and can look up (and hopefully fix) issues. In the support of Postgres, or any true community open source product, different companies can provide support. If you don’t like the company you work with… you switch. This drives these companies to be really good at delivering support! How is that for an eye opener.

Extensibility

One of the superb advantages of Postgres is its native extensibility. I mean, think about it for a moment… having a relational database platform with the strength of Postgres, the strength of Oracle or Microsoft SQL Server for that matter. Postgres gives you more options to integrate a wealth of data sources, data types, custom operators and many more other extensions than you will ever need! The integration into Postgres is so solid, these extensions function like any other function in the core of Postgres.
And, rest assured, chances that you will ever be faced with having to built this yourself are extremely slim. There are 30 to 40 thousand developers working with larger and smaller pieces of code of Postgres. Chances are that if you find yourself challenged, somebody else faced and solved that challenge before you. That solution will then be available for you to take and use, solve your challenge and move on. That is also open source for you.

This capability is what makes Postgres ultimately suited to fit the central role in any polyglot environment, we see being built today.
Maximizing the amount of information from data available in multiple data silos in an organization. This is a challenge we see more and more often today. Integrating traditional  applications as ERP, CRM, with data-warehousing results, again combined with Big-data analysis and event-data-capture aggregates. This generates additional decision-driving information out of the combination of these silos. Postgres, by design, is ultimately suited for this. It saves you for migrating YUGE amounts of data from one store to another, just to make good use of it.
The open source Dogma “Horses for courses” eliminates double investments, large data migrations or transformations, it just enables you to combine and learn from what you already have.

End of part II

A link to part three of this blog post will be placed here shortly.

Why I picked Postgres over Oracle, part I

As with many stories, if you have something to tell, it quickly takes up a lot of space. Therefore this will be a series of blog posts on Postgres and a bit of Oracle. It will be a short series, though…

Let’s begin

History

 I have started with databases quite early on in my career. RMS by Datapoint… was it really a database? Well, at least sort of. It held data in a central storage, but it was a typical serial “database”. Interestingly enough, some of this stuff is maintained up to today (talk about longevity!)
After switching to a more novel system, we adopted DEC (Digital Equipment Corporation) VAX, VMS and Micro VAX systems! Arguably still the best operating system around… In any case, it brought us the ability to run, the only valid alternative for a database around, Oracle. With a shining Oracle version 6.2 soon to be replaced by version 7.3.4. Okay, truth be told, at that time I wasn’t really that deep into databases, so much of the significance was added later. My primary focus was on getting the job done, serving the business in making people better. Still working with SQL and analyzing data soon became one of my hobbies.
From administering databases, I did a broad range of things, but always looping back to or staying connected to software and software development using databases.
Really, is there any other way, I mean, building software without using some kind of database?
At a good point in time, we were developing software using the super-trendy client-server concept. It served us well at the time and fitted the dogma of those days. No problems whatsoever. We were running our application on “fairly big boxes” for our customers (eg. single or double core HP D 3000 servers) licensed through 1 or 2 Oracle Database Standard Edition One licenses, and the client software was free anyway…
Some rain must fall
The first disconnect I experienced with licensed software was that time we needed to deploy Oracle Reports Server.
After porting our application successfully to some kind of pre-APEX framework, we needed to continue our printing facilities as before. The conclusion was to use Oracle Reports Server, which we could call to fulfill the exact same functionality as the original client-server printing agent (rwrbe60.exe, I’ll never forget) did. There was only no way we could do this, other than buying licenses for (I though it was) Oracle BI Publisher, something each of our clients had to do. This made printing more expensive than the entire database-setup, almost even the biggest part of the entire TCO of our product, which makes no sense at all.

More recently

This disconnect was the first one. Moving forward I noticed and felt more and more of a disconnect between Oracle and, what I like to call, core technology. Call me what you will, I feel that if you want to bring a database to the market and want to stay on top of your game, your focus needs to be at least seriously fixed on that database.

Instead we saw ever more focus for “non-core” technology. Oracle Fusion, Oracle Applications (okay, Oracle Apps had been there always), and as time progressed, the dilution became ever greater. I grew more and more in the belief that Oracle didn’t want to be that Database Company anymore (which proved to be true in the), but it was tough for me to believe. Here I was, having spent most of my active career focused on this technology, and now it was derailing (as it felt to me).

We saw those final things, with the elimination of Oracle Standard Edition One, basically forcing an entire contingent of their customers either out (too expensive) or up (invest in Oracle Standard Edition Two, and deal with more cost for less functionality). What appeared to be a good thing, ended up leaving a bad taste in the mouth.
And, of course… the Oracle Cloud, I am not even going to discuss that in this blog post, sorry.

The switch to Postgres

For me the switch was in two stages. First, there was this situation that I was looking for something to do… I had completed my challenge and, through a good friend, ran into the kind people of EnterpriseDB. A company I only had little knowledge of doing stuff for PostgreSQL (or Postgres if you like, please, no Postgré or things alike please, find more about the project here), a database I had not so much more knowledge of. But, their challenge was very interesting! Grow and show Postgres and the good things it brings to the market.

Before I knew it, I was studying Postgres and all the things that Postgres brings. Which was easy enough in the end, as the internal workings and structures of Postgres and Oracle do not differ that much. I decided to do a presentation on the differences between Postgres and Oracle in Riga. I was kindly accepted by the committee even when I told them, my original submission had changed!
A very good experience, even today, but with an unaccepted consequence. -> The second part of the switch was Oracle’s decision to cut me out of the Oracle ACE program.

It does free me up, somehow, to help database users across Europe, re-evaluate their Oracle buy-in and lock-in. Look at smarter and (much) more (cost)-effective ways to handle their database workloads. This finalized “the switch”, so to speak.
Meanwhile more and more people are realizing that there actually are valid alternatives to the Oracle database. After the adoption of the Oracle database as the only serious solution back in the early 1990’s, the world has changed, also for serious database applications!

End of Part I

Please follow this link to the second part of this blog post.

Open Source? We have been here before… right!

Since over half a year I have made an adjustment in my course… nothing too dramatic, but still it has had some impact.

I have chosen the path of the more pure technique again. Not in a sense that I don’t manage anymore, though I don’t, but that’s more of a side effect
I have chosen the path of the more pure technique in a sense that the change from Closed Source software to Open Source software allows you to actually work with and solve things by creating solutions rather than trying to figure out how something, someone created for some issue, can reconfigured so it resembles a solution for your actual problem.

Okay… okay… this of course is exaggerated, but it serves to help think about the issue.

No one ever got fired for buying Oracle” is one of the phrases I have heard numerous times over the past period.
Well, no… but it’s also no free pass to –sorry for the phrase– waste money on technology you either never going to use.

Over 80% of the installed base uses less than
20% of the technology they are paying for!

I have followed a number of the brightest mind in the industry (our industry, the database industry) for many years investing vast amounts of time in reverse-engineering pieces of technology that have been built, in order to explain certain behavior.
Of course, very necessary, no argument there, but wouldn’t it be so much more cool if this overwhelming amount of brain-power could be used to actually create stuff??

Open Source in stead of Closed Source…
The answer, I think!

Sure, I am raised with vendor created solutions, that was the default MO when I got trained. VMS, MS-DOS, HP-UX… (are you _that_ old, yes, I am _that_ old) and a number of applications that did the work.
Well, those days are gone… operating systems in data centers have (nearly) all been replaced by Linux distribution installs. And I mean like, as good as all of them.

Sufficiently stable, cost effective and they get the job done.

Next wave?
Databases!

With the current explosive growth of Open Source databases, brace yourselves. Or rather, embrace!

All the exact same arguments that are there for Operating Systems apply. There is no difference, and you, the industry, chose! And you will choose this again. Simply because “it is good enough”, it is much more cost effective and it gets the job done.
The extensibility, the agility of Open Source database software gives you the ability to let your database, be it OLTP, OLAP, Big Data, Polyglot, or whatever we come up with, do what needs to be done.
The current leader of the Open Source relational database systems is PostgreSQL. A platform developed in over 30 years to become an absolutely stable data processing engine for a fraction of the costs of the Closed Source players in this market.

Conclusion:

  1. We have seen wave 1 of Open Source where we all choose to replace the operating system standard with Linux.
  2. We will now see wave 2 of Open Source where we will choose to replace the database management system standard with… PostgreSQL (or in specific cases one of the other, more specialized systems, depending on the need).
Hope this helps!

Birth of a user group conference…

Or, how post-conference blues hit.

Seeding

You don’t actually get to witness these kinds of things to often. Yes, there was the birth of POUG conference, the passionate work of Kamil Stawiarski and the people he gathered around him. He did an awesome job and pulled it off.
Why then is this so special? Well, first of all because it is my native conference. People that have contributed for many years, working closely together with new members to create something new. I think it is kinda special.
Richard Olrichs, Ise Douwes, Luc Bors and Bart van de Laar formed the team that pulled it of! Kudos to them.

Work started already end of 2016… on the notion that this conference was being organized, there was a small Twitter bombardment to recruit as many of the interesting international speakers to come and join us. This helped create a fabulous agenda, covering 81 (!!) sessions and 3 keynotes over 2 days in the spectacular setting of “De Rijtuigenloods” in Amersfoort.

Importance for NL

We have (finally) done it! The Netherlands have experienced their very first Full Stack Oracle Conference ever! I have said this many times before, and I will say this probably many times again, this is so very important for the spread of knowledge, the exchange of experiences and cross-pollination between countries!
We have never done this before… We have APEX World, which is, of course, super important! And we have SIGs, which are very important for people working within a specialization. All good, all very important! But our business is way to specialized already. If you never take the time to look over your boundaries to what your colleague is doing (for which you don’t have time on a day-to-day basis) you’ll get isolated and miss out on possible great idea’s, changes and inspiration. For this alone, events like these are so important. For a country / region (as we span Benelux) that is so active in IT, it is a nuisance to have to go to either the UK or to Germany to experience a conference like this.

It is _that_ important…

One personal thing… nl.OUG (this is the brand new name for OGh, which also symbolizes a new start to me) focus on talks on (end-) user experience, in effect, partners with end-users coming to share project briefs… Good in itself, but not what I would applaud as main focus. These conferences – to me – are about learning and the best learning comes from professionals sharing either best practices or telling about technical implementations of technique. These stories are obviously always very welcome, but are no main focus…

My personal experiences

— International scene

As an active member of the Oracle community, I tend to know a number of people, spread out over this globe. One of the joys of a general conference is the fact that many of these people also participate. This leads to many happy encounters. With Oren Nakdimon from Israel, to Sandra Flores from Mexico, with Tim Hall, aka. Oracle Base, Maria Colgan & Brendan Tierney and therefor with Chris Saxon, more than 2rds of the Ask TOM-team!! And even many more famous speakers from home and abroad. It was a very special feeling to meet all these beautiful people on my home turf!

— Followed sessions

I didn’t get to follow many sessions, partly because of the many people I met and wanted to catch up with and partly because of, well, other responsibilities…
And perhaps a bit because of the fact that the ratio between serial sessions and parallel sessions was a bit off.
I did get to see:
The keynote by Maria Colgan highlighting the many things you can do with an Oracle database
Investigating the performance of a statement via the SQL Monitor report by Toon Koppelaars, which is always insightful!
Moving Oracle data in real-time – The 3 fundamental principles of Oracle replication by Jakub Sjeba from Dbvisit, which proved to be an excellent basis for my own session, later that day
Blockchain on the Oracle Cloud, the next big thing by Robert van Mölken, who helped me understand the technical side of the Blockchain technology.
It’s a wrap by Lucas Jellema, who did a great job at really zooming out and looking at the bigger picture.

— My sessions

I had the lucky opportunity to present even 2 sessions in Amersfoort.
Migrating your Oracle database with almost zero downtime
and
Comparing PostgreSQL to Oracle

Both sessions were well attended and interactive. I enjoyed it very much and, judging from the reactions and interactions, I guess the attendees too. Thank you for attending!!
Obviously I am happy to see the uptake of PostgreSQL and EDB Postgres in the Benelux. As said with “horses for courses”, Oracle has it’s playing-field, but so does PostgreSQL, and probably bigger than it is today 🙂

And now, the future…

This was 2017, this was the kick-off, the very first one.
With the buzz and with the post-conference blues…

It now is time to look to 2018, start preparing. Gathering lessons learned, inventorise feedback and make plans.
Whatever the outcome, there can only be 1 plan! “nl.OUGTech18″ or “Tech Experience 2018” we need to make sure the messages reaches further and wider.
With over 250 attendees for a first event, we aim for over 500 for the next event. There are more than enough potential participants in our region to pull this of.
The basic structure is there, the first succes is there, let’s do this!!

See you all next year!!
(or hopefully sooner)

Hey JAVA-developer, why don’t you love your database

Why this post?

Partly, this blogpost is a result of a promise to Lukas Eder. Basically my vision adheres quite nicely to the “Thick Database” driven by Bryn Llewellyn and Toon Koppelaars who, understandibly, drive this from an Oracle perspective.
It –more than of course- also nicely fits EnterpriseDB or even vanilla PostgreSQL database landscapes.

There is apparently still so much confusion in the world on the how, why and what of good application development and architecture that I decided to chip in my bit. I think I have a bit of an idea on how this aught to work and I also think it is not a half bad idea, plus a couple of people whom I highly regard, seem to agree with it. So here goes…

Traditionally

Traditionally there is no big love between application developers and their persistence-store. I don’t really know why because I never found the opportunity to do a real inquiry, but I think I have a reasonable understanding.

Basically there is constantly the enormous pressure of delivering new features and functionality. So much even that the basic development work, the more “boring” and “time consuming” things -why pay now, if you can also pay later- get postponed. Things like peer-reviews, (integration) testing, technical design… Basically, more people means more features.
If even these things get too little attention, why would something like an overpriced library-box get more attention? Not to mention these DBA’s you need to pass to even get close(r) to this library-box…

Here are my four reasons why I think
it should deserve a chance!

1. Easier

Plain and simple. It is easier. If you take a structured query language like PL/(pg)SQL, it is basically easy. Founded on the programming language ADA, it is easy to understand and one can quite easily build a number of routines, procedures, packages, functions to let the database chop and glue data and just deliver the results for your application to consume.
It saves you the time of having to (re)write some of this more complex stuff in your application or perhaps even over a several applications.

2. Quicker

As said under the first point: build once, use many times. By creating mechanisms in the database, you get the opportunity to think about the separating data manipulation mechanisms from data representation mechanisms and where you want to put which specific function. Of course, this decision process takes a little extra time in the beginning, but will repay many times over as your projects grows and gains meaningful complexity (is there something like “meaningful complexity”, well, yes…).

Quicker also is in operational response times. Querying a stored procedure will bring agility to your application in a sense that this stored procedure will be much quicker in getting you the answer than if you do this in a distributed (middle-ware) environment. These stored procedures can be accessed through REST-endpoints, giving flexibility and the possibile desired disconnect between the database and the application layer.

3. Cheaper

Powerful, because you are and you remain close to the data. No data transportation overhead, no latency. These kinds of slimming down, mean less requirements to infrastructure and distributed capacities (either hardware or (virtual) “cloud infrastructure”. This slimming down frees up budget which can then be spend on the more meaningful bits and pieces of your application.

4. More consistent

Finally, I think this approach brings more consistency. As you do things near the data it lives on, near the processing power you depend on, you also get single access paths to specific bits of your data, to specific constructs that drive the value of the application you are building. Through this, it does not matter from where you call this service, REST-endpoint, stored procedure, or whatever you call it, you always get the same answer, driving a consistent decision process based on that application.

And if ever something changes, there is also just one place for you to ensure these new requirements are added. Voila, consistency throughout your application landscape, as all that depends on this data-set gets this uniquely updated information.

The magic of working together

There is a lot of misunderstanding between DBA’s and Developers. I have been in both roles at some point and I have seen this happen first hand. One of the things, though, in that force field, I have learned, is the power, the joy and, through this, the magic of working together.
In the end, we all have the same goal, which is furthering our business by being the best at what we do. This means, for a developer, meeting feature request, short development cycles, quick delivery and as much as possible, get it right in one go. For a DBA this means making sure the database stays consistent, performant and available. And, in extension of that, for operations it means that the final product must be easily and quickly deployable to enter into an uneventful and dependable life-cycle.

Bringing together these seemingly conflicting disciplines is fun! By investing a little time in exploring the other disciplines, you will find common drivers, in a sense that everyone want the same thing. By getting over nearly religious initial differences, you will find magic in the combination. You will reach your goals earlier with the bonus that your co-workers will also reach their goals earlier and have a better end result than you dreamt possible.

True JAVA-Champion

Coming from another world, I do not know the requirements for becoming a JAVA-Champion. I imagine it to be not too much different from other recognition programs out there… But…
If you create more features and functions and you are able to run your application with greater concurrency on (way) less platform-power. Thus increasing the RIO on your application, this makes you a true Champion of JAVA and your business!! If you are able to combine this with some magic in your cooperation’s…
Believe me, it is more fun in the end too!

Riga Dev Days 2017, new experiences in many ways.

Riga Dev Days 2017

General

It has been a while since my last blog-post.
One of the reasons is my shift from closed to open source software, databases more specifically. More on that in a later blog-post.

The reason for already mentioning this is this strange hybrid (what a popular word, these days) situation that I am in at the moment.
Thanks to the super enthusiastic, flexible and tenacious organization-team of the Riga Dev Days, I was able to participate.
Happily boarded the Air Baltic flight, I went on my way to Riga!!

Being new at the broader conference scene, I enjoyed being at a mixed source developer conference. Besides the usual suspects – some of which are my best friends – I got to meet many interesting new people.
One of the key phrases of the day is: “the more you learn, the more you realize you know nothing – John Snow…” and it’s true! You never stand to think about it, but the wealth of subjects is just tremendous and the combined knowledge at events like these is down right “Yuge, it’s awesome, tremendous!”

Day one

With a day like this, time flies. Between session (and during sessions) there are discussions, a bit of work and catching up to do.
Still I managed to catch a few sessions, like the one from Michael Hüttermann who made a clear and well rounded case regarding CI/CD in a DevOps world. A nice insight into the effort that goes into what’s behind the proverbial “push of a button”.
Another example was that by Marcos Placona about the many (and very basic) things that you have to keep in mind wen building apps. There is no silver bullet and the best you can achieve is to discourage the hacker so much, they move on. Much like securing your house, do to speak.

The day ended in the medieval basements of Riga, where we had some really good medieval food. Life is good…, well…, it has it’s moments!

Day two

The keynote address by Edson Yanaga, which kicked off day two of the Riga Dev Days, was quite interesting.
Shortening development and deployment cycles and shrinking feature release sets actually helps improving software and deployment quality by creating faster and more accurate feedback loops. By looking at these concept in this way, buzzez like DevOps and Agile actually get some hands and feet. One of the lessons, though, is that doing things this way do not eliminate work or automagically solve various issues for you! It will help in getting predictability and continuity into your software development processes.
A nice eye-opening remark finally, was… “no, I don’t pay you to make something work on your computer, I pay you to make something work on my computer(s)!!”

Another talk I was able to attend was around Blockchains. Something I knew nothing of and was actually quite interested in. Nick Zeeb took us through a very lively and very animated tour of what actually a Blockchain is and what the awesome potential of this technology can be. I was impressed.

With this, the second day draw to and end and therewith also my turn “in the pit”. As this event is held in a movie-theater, every room had a sloped tribune, which was often packed with enthusiastic participants. I had the opportunity to share my thoughts on the comparison between PostgreSQL and Oracle.
The session was very well attended with a lot of questions regarding the possibilities of using these other technologies in scales that were not really considered before. You can find a recording of the actual presentation here as soon as it comes available.

Riga Dev Days was a good conference. I would recommend everyone to either attend or submit an abstract for their event in 2018!!