Neues vom PostgreSQL Planet

Why do we need vector databases? The proliferation of embeddings immediately brought forth the need to efficiently store, index, and search these arrays of floats. However, these steps are just a small piece of the overall technology stack required to make use of embeddings. The task of transforming source data to embeddings and the serving of the transformer models that make this happen is often left as a task to the application developer. If that developer is part of a large organization, they might have a machine learning or data engineering team to help them.

Coming up this week: the fourth installment of the Postgres extension ecosystem mini-summit. The topic: Trusted Language Extensions, a.k.a. TLEs.

Any database engine, including PostgreSQL, requires some routine maintenance to ensure optimal performance. Especially when there are massive delete or purge operations to delete rows from a Table, the fragmentation builds up within a table. Reclaim this fragmented space from disk requires a table to be rebuilt. At HexaCluster, we had many Customers asking us […]

Toulouse, 15 April 2024

The upcoming PostgreSQL 17 will ship with improved query cancellation capabilities, as part of the libpq, and so will the upcoming release of Psycopg version 3.2.

About a month ago, Alvaro Herrera committed the following patch to PostgreSQL:

A new feature to run a single script at the very beginning of the cluster lifecycle.

pgenv: run once scripts

Today pgenv got a new release that provides a simple, but quite useful, feature: the capability to run a custom script the first time the instance is started.

Recently I joined Ry Walker, CEO of Tembo, as a guest on the Hacking Postgres podcast.

Hacking Postgres has had a lot of great Postgres contributors as guests on the show, so I was honored to be a part of it being that my contributions are more in the form of developer education and advocacy.

Ry asked me about when I got started with PostgreSQL and what my role looks like today.

I just got the coin in the mail!

PostgreSQL 16 Coin

I just received in the mail the PostgreSQL 16 Coin with a great artwork!

I’m really happy to be part of this great community!

Have you ever wanted to use a non-superuser role in a Postgres database to perform actions that are normally restricted? Even something as simple as reading from the pg_stat_activity view requires special permissions to view the query column because it could contain sensitive information.

This is not the usual Waiting for post, but something should be said. Back in March/April of 2022 Andres Dunstan committed a series of patches that added support for lots of really interesting features from SQL/JSON standard. While I'm not avid user of json in database, I was very, very happy. Wrote couple of blogposts … Continue reading "Waiting for …: SQL/JSON is coming back. Hopefully."

Databases Cursors are one of the key components of development. We have been using them for traversing query results for processing based on business logic or functionality. In Code Conversion, Cursor code always needs to be ported either from PL\SQL or TSQL to Pl\pgSQL, and some special considerations need to be incorporated as best practices to avoid surprises or functionality failures.

On 3rd of April 2024, Tom Lane committed patch: Invent SERIALIZE option for EXPLAIN.   EXPLAIN (ANALYZE, SERIALIZE) allows collection of statistics about the volume of data emitted by a query, as well as the time taken to convert the data to the on-the-wire format. Previously there was no way to investigate this without actually … Continue reading "Waiting for PostgreSQL 17 – Invent SERIALIZE option for EXPLAIN."

On 1st of April 2024, Masahiko Sawada committed patch: Add new COPY option LOG_VERBOSITY.   This commit adds a new COPY option LOG_VERBOSITY, which controls the amount of messages emitted during processing. Valid values are 'default' and 'verbose'.   This is currently used in COPY FROM when ON_ERROR option is set to ignore. If 'verbose' … Continue reading "Waiting for PostgreSQL 17 – Add new COPY option LOG_VERBOSITY."

On 30th of March 2024, Dean Rasheed committed patch: Add support for MERGE ... WHEN NOT MATCHED BY SOURCE.   This allows MERGE commands to include WHEN NOT MATCHED BY SOURCE actions, which operate on rows that exist in the target relation, but not in the data source. These actions can execute UPDATE, DELETE, or … Continue reading "Waiting for PostgreSQL 17 – Add support for MERGE … WHEN NOT MATCHED BY SOURCE."

When you first spin up your app, the emphasis is on getting started and getting data to your clients. But when you don’t have throughput, you are also not going to have enough concurrency to unveil bad queries.

But then you have success. And success means data. More users, more interactions, more everything. Suddenly, queries that performed fine are struggling under the load, hurting performance and scalability. This is all going to mean a far worse user experience when much higher costs because of inefficient resource use.

PeerDB offers a fast and cost-effective way to move data from PostgreSQL to data warehouses, such as Snowflake, and to queues like Kafka. This enables businesses to have real-time and reliable access to data, which is of utmost importance in this AI ...

Last October, I wrote:

Looking at the numbers in the PG17 cycle so far, even though they are much lower than the 2023-03 commitfest, we can expect that PG17 will have more activity in total than previous development cycles.

And now it’s time to count up the score. The final per-commitfest numbers of committed patches were:

A guide to Postgres roles and privileges

Terrific presentation and discussion at last week’s Postgres Extension Ecosystem Mini-Summit. In fact, I later learned that some people couldn’t make it because the Eventbrite event page showed it as sold out!

I recently created a Postgres extension which utilizes the pg_cron extension to schedule recurring activities using the cron.schedule(). Everything worked great. The only problem was when I dropped my extension, it left the cron job scheduled, which resulted in regular errors:

One fun fact of Pl\pgSQL is that it’s an Extension in PostgreSQL that is created as default whenever we create any new databases.

Converting databases, primarily those like Oracle, involves translating multiple functions, procedures, or packages written in PL/SQL to PostgreSQL with PL/pgSQL.During the conversion stage, we utilize conversion tools to automate a significant portion of the initial migration. However, it’s important to note that automatically converted code might not always follow best practices for PostgreSQL development.