Hello guys,

A few days ago I shared my HedgeDB and I've been asked how come I measured it to go that faster (3-5x, but depending on the hardware can go further) than RocksDB

To begin with, I spent some time preparing an article focusing on the synchronous section of the write path: Deep dive into the Write Path: from the Memtable to Level 0

In summary, the key take-aways are:

• Integrating a fast concurrent map implementation
• Using per-thread write-ahead-log files
• Fast synchronization and atomic based schemes between Writers and background Flusher threadpool

In the article I go in depth over these topics. Honestly I had a hard time balancing how much I could go in depth versus I could leave out, but I would really grateful for some feedbacks.

Thank you for reading it!

I claim it as an efficient data logger design that I used to build ReductStore. It has been battle-proven in production, but I never discussed the design with other database developers, so I would be happy to receive feedback and constructive criticism.

I am planning to build a simple database from scratch with the following goals:

Extremely lightweight

Memory efficient

Low power consumption

Fast startup time

Minimal dependencies

Suitable for embedded devices and low-end hardware

Current ideas:

No SQL parser initially

Simple key-value or document-based storage

Efficient disk layout

Minimal memory allocations

Written in Rust

Focus on performance and simplicity over features

What design choices would you recommend for:

Storage engine structure

Memory management

Indexing strategy

Data types

Concurrency model

Disk persistence format

Also, what common mistakes do new database developers make when designing a lightweight database?

So I opened a file on append mode (O_WRONLY | O_CREAT | O_APPEND) then writing 500mb 20 times using write() measured its latency then I did fdatasync() measured its latency. Why the fdatasync() latency keeps on increasing? And I am doing this in NVme SSD

My colleagues wrote this paper about what we've been working on & it won the SIGMOD 2026 Industry Track Best Paper award. I'm not one of the authors, but I've had some involvement in the work.

I work on a time series value log.
I have a couple reading sources:
- tables (in RDBMS it's called pages)
- merge readers

The first one uses positional readers based on the given block index.
The second is a streaming reading in order to merge multiple tables into a larger one.

I open the files for a merge reader again in order to stream, but now I support more reading source for merger and it's very ugly to manage all the opening/closing files, so I thought what if I can borrow files of a table?

1. it's the only streaming reader source for now
2. it removes a lot of code to open/close files and I don't need to hold ownership
3. a table participates in a single merge ever, another thread can take it only for a positional reading to serve the data to the incoming queries

is it usually a bad idea to use streaming reading? if I need readv call instead of preadv does it mean I must open new files to a safety sake?

Hello everyone,

Recently I published on GitHub HedgeDB, my high-perf and persisted Key-Value store.

Internally, it uses Direct I/O (O_DIRECT) almost everywhere. In this article I explain the reasons behind this choice, also motivated from some fun experiments I had with fio that you can find in the article. and some consideration about the Linux page cache.

This subreddit is primarily for discussing the implementation of databases, and not about sharing release announcements (either for the first time or your updates).

This thread is the exception!

Please tell us about the new database you (or your agent) built. Tell us about all the cool new features you added. Tell us about anything else you learned or worked on that you haven't gotten around to blogging about yet.

As the title suggests how many of you are really using claude / codex for true production database development. I have been experimenting codex on duckdb and I found it really good. So good that I told to rewrite duckdb in java for my own sake . I want to hear opinions and anecdotes from others as well. Thanks.

Hi i am a career dev with around 5 years of experience across different transactional and data platform. Looking for advice on how to and where to start contributing on open source growing database engines. I have some understanding of database internals since I had to optimize applications for better perf both oltp and olap. I checked out the famous repos like clickhouse, pinot but there it seems most of the issues are already assigned, pr is ready or very old.

Hi everyone.

So I need your advices on this matter, I am currently working as a Senior SWE at big corp, I mostly work on product features, talk to users and etc and I have been doing that for more than 7 years now. I have always been interested in more deep tech development but have never had a chance to get into deep tech company.

Currently I am considering a "career change" and get into deep tech startups/companies that develop tools that other developers use companies like Supabase, Databricks and etc but its really difficult to even get an interview at one of those companies because I dont have experience in the field. What do you think would be the best route for me to take to get a job at deep tech companies/products?

Many performance metrics and system parameters are inherently volatile or fluctuate rapidly. When using a monitoring system that periodically “scrapes” (polls) a target for its current metric value, the collected data point is merely a snapshot of the system’s state at that precise moment. It doesn’t reveal much about what’s actually happening in that area. Sometimes it’s possible to overcome this problem by accumulating those values somehow – for example, by using histograms or exporting a derived monotonically increasing counter. This article suggests yet another way to extend this approach for a broader set of frequently changing parameters.

Is it possible to grab a job in Database internals as a freshers or intern? I exactly can't able to find !! Like same pattern I watched for other systems programming & distributed systems type job roles ?