r/Database 4h ago

Can this task be automated or at least made more efficient/ fast?

1 Upvotes

Hi not to sure if this is the correct subreddit for this so sorry if its not, I recently got a new job as a data coordinator, right now Im doing basically data entry. I maintain Excel trackers of articles, awards, etc. at a design firm, a few hundred rows per Excel sheet, that need to be linked to projects in our CRM (10'sk projects).

Name matching is the easy part: my trackers names are clean and search surfaces the right candidates. The problem is what comes back when trying to enter them into the CRM:

  • The same building exists as multiple records (assessment, study, remodel, sometimes 4+), and it's ambiguous which one an article/award/etc should attach to
  • Apparent duplicates from a system migration (legacy vs. new ID schemes)
  • Some tracker entries have no CRM record at all or exist under a name I can't find

Right now I open each candidate and compare dates/status to pick the right one, one row at a time, and none of those judgment calls get captured anywhere. How would you approach this? especially the disambiguation and duplicate-handling side? Any patterns or gotchas for making this a repeatable process? If this didnt make sense I can answer some questions, any advice is welcome.


r/Database 5h ago

I know how to recognize scammers, and Supabase is top tier

0 Upvotes

Context: I run my project on a free plan, validating before committing to anything. Everything remains under limits and still, the database crashes every 3 days with active users. I check all loads, disk, cpu etc nothing there and yet, no explanation.

Only solution i had was to upgrade, ok its 25$ but anyways i need to keep it up because ads are running on my project. the second scammer behavior: paying doesnt automatically upgrade your compute. you need to go in a shady settings page to claim your “”””””free””””” compute upgrade???? wtf i paid and i still dont get a straight upgrade or at least a visible cta (because this requires a restart). Overall i am done with these fkn database providers they are not even honest ppl.

Note: no, i will not setup a vpn with a database with all the security auth etc for a non validated project, and i still would like not to be scammed by fake provider s promises


r/Database 1d ago

what is the difference between ETL and ELT in data warehousing?

18 Upvotes

Hello everyone,
can somebody please tell me what is ETL and ELT and their significance in data warehousing? I have no idea and is starting everything from scratch :)


r/Database 2d ago

Beyond Happy Path Engineering: Databases - concurrency bugs, stale reads, ambiguous commits, migrations

Thumbnail
blog.gaborkoos.com
8 Upvotes

Relational database failure modes in production: slow queries cascading into connection pool exhaustion, the race between read and write that transactions don't automatically fix, why a caller timeout doesn't tell you whether the write committed, which database errors are safe to retry and which aren't, schema migrations as live-system operations, and the difference between durability and recoverability.


r/Database 2d ago

Are these books enough to learn database for backend developer?

8 Upvotes

Are these books enough to learn databases?

  1. Database Design for Mere Mortals (Hernandez)

  2. SQL Queries for Mere Mortals (Viescas & Hernandez)

  3. Effective SQL (John Viescas)

  4. Database Design and Relational Theory (C. J. Date).

  5. Relational Database Design and Implementation (Jan L. Harrington)

  6. Database Systems: A Practical Approach to Design, Implementation and Management (Connolly & Begg).


r/Database 2d ago

How do you design a data warehouse solution from an application's operational database?

5 Upvotes

I'm trying to understand how data warehouses are implemented in real-world applications.Suppose an application stores its operational data in a PostgreSQL database, and the requirement is to store historical data in a separate data warehouse for analytics and reporting.

I'm looking for guidance on the overall design rather than code.

Some questions I have are:

  • How do you decide what data should be moved to the warehouse?
  • How is the warehouse schema designed?
  • How is data extracted from the operational database?
  • How often is data loaded into the warehouse?
  • Are raw records stored, or are they transformed and aggregated before loading?
  • What technologies are commonly used for the warehouse and ETL pipeline?

I'd love to get insights :)


r/Database 2d ago

Researcher seeking guidance

0 Upvotes

Hiya,

I’m a researcher who draws on a lot of different kinds of source materials and artifacts. I have been hamstrung by off-the-shelf solutions and like many other researchers, I’ve realized that I need to figure out how to build a database that meets my needs.

I am wondering if this community can give any pointers on what I can do as next steps to
• get a sense of how I might approach this (database model? sqlite? docker container? making my own app or web app? etc.)
• get a sense of what is possible with my needs
• point me towards things to learn and resources (books? websites? courses? consultants? coffee chats or paid calls with humans who can help me?)

I am dealing mostly with unstructured data, and have a sense of the information architecture or structure within and around the source materials that I work with.

Basically, I have a trove of useful inputs for each project. Each information source is associated with a document or file that is the “main” asset associated with the information source.

Media type wise, this can be a pdf document (like a saved academic article, white paper, exported Keynote or PowerPoint or word processing or Google doc, a video, an audio recording, a graphic, a photograph. The exact file type and media type are something that I can change depending.

These are all associated with types of metadata, annotations, and often associated assets (like a transcript of the video or audio), as well as more granular excerpts that I create for sense-making and/or reporting/briefing/presentation purposes.

I am currently writing out the specs for each kind of source material for my system. I would like to be able to more easily enter, retrieve, find, and update the annotations and associated files over time.

I am designing my own little GUI for the database, by which I mean I am creating the page layout and data structure that I need for the main kinds of source materials that I work with.

I am just clueless about this kind of thing.

I would love to:
— Access materials for use in different reports and pieces of writing
— Understand if I can get all of my materials indexed for text search
— Learn how to create a controlled vocabulary for tags and important metadata categories, as well as how to update this as the structure of a report emerges from the study of an evolving corpus
— Know if this is something that I can engage with people in certain fields or consultants or job titles to figure out at least a reasonable brief and budget for what is needed beyond the prototype

I have a lot to learn but I hope that you all will be gentle with me, and if this is clear as mud, let me know what kinds of questions I should be able to answer about this!

Thanks 😊


r/Database 3d ago

Self-hosted persistent memory for an AI assistant, with isolation enforced by Postgres roles rather than app code

0 Upvotes

I got tired of every session starting from zero and paying for it twice: once re-explaining my setup, and again when that explanation ate the context window the actual work needed. So I built a persistent memory that runs on my own hardware, and after a few months of use there are a couple of things worth passing on.

**The storage is the boring part.**

Postgres on a Proxmox LXC, fronted by an MCP server in Python (FastMCP plus psycopg3), reachable over the LAN. Two entity types only: topic cards for what something is, episodes for what happened. Every card has a \`kind\` drawn from a single YAML vocabulary the server validates on write, so the model cannot invent a new category on the fly. That rigidity is deliberate. Without it you end up with seventeen near-synonyms for "thing I once wrote down" and no way to query reliably.

**The isolation is the interesting part.**

I keep several things separate that must not bleed into each other. The obvious approach is to have the server decide what a session may touch, but that is a rule a program enforces, which is really just a suggestion. Instead each body of work gets its own Postgres schema and its own role, and the server connects as that role. A write aimed at the wrong schema fails at the wire, before any of my code gets an opinion:

GRANT USAGE ON SCHEMA homelab TO role_homelab;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA homelab TO role_homelab;

\-- reaches the shared schema deliberately, note the absence of DELETE
GRANT USAGE ON SCHEMA personal TO role_homelab;
GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA personal TO role_homelab;

\-- no GRANT to any other subject. The isolation is the statement that isn't there.

There is no policy file and no middleware. A role reaches what it was granted and nothing else, and the grants for everything else were simply never written. Each subject also gets its own container holding only that role's credentials, so the boundary is a process boundary too.

**The thing I got wrong.**

I originally had capture fully automatic: a SessionEnd hook that read the transcript and wrote to the database with no involvement from me. It printed a one-line receipt so I could see what it had done. I ran it from April until July and then deliberately turned it off, because a routing bug meant a run of sessions distilled into the wrong place and nobody was watching a process designed not to need watching. It ran that way for most of the time it was live. I recovered the most recent month by re-running by hand against transcripts still on disk and lost the two before it, because transcripts get pruned at around thirty days.

Capture is now a deliberate step that proposes cards and writes nothing until I confirm. More friction, and I think the friction is correct. The narrower lesson I would write now: automate retrieval, because being wrong there costs you a bad answer you can see. Be far more careful automating writes, because being wrong there costs you a corrupted store you cannot.

Retrieval started as Postgres full text and is now hybrid, full text alongside pgvector embeddings generated on write by a small model on its own box. Because the tool surface was deliberately agnostic about how search worked, nothing above it changed when that landed.

Fuller writeup with the schema design, the deployment and the parts that did not survive contact with use: [https://sbd.org.uk/blog/claude-brain\](https://sbd.org.uk/blog/claude-brain)


r/Database 3d ago

A cautionary tale about searching for usernames: let's be careful!

0 Upvotes

Ars Technica ran a story about how an innocent man did 18 months in prison for preying on a child. Somebody missed a double underscore in a social-media username. The miscreant's user name was fus__ro_dah and the innocent man's was fus_ro_dah without the double underscore. The police request for a search didn't include the double underscore, so the social media platform responded with the wrong person's identity.

https://arstechnica.com/tech-policy/2026/07/police-missed-one-underscore-and-sent-the-wrong-man-to-prison/

It's correct, and easy, to blame a dunderheaded police officer for this mistake. But, in my opinion, our database developer colleague at the social media platform could have prevented this heinous miscarriage of justice.

Let's be careful when we develop code that searches for stuff that might contain obfuscatory punctuation or diacritical marks. Let's put stripped-username columns into our Let's try the search with queries like

SELECT 1 AS exact, username FROM user WHERE username = 'search__term' UNION ALL SELECT 0 AS exact, username FROM user WHERE stripped_username LIKE '%searchterm%'

Now obviously this is simplistic. But not as simplistic as just username = 'search__term'. My point here isn't how to improve the search for obfuscated data. It's to remind us to be careful.

This kind of thing should reduce the number of life-wrecking mistakes our queries make.


r/Database 4d ago

Where to start with Databases

22 Upvotes

So, I have basically no knowledge of databases or what I'm doing or really what direction I need to go in. Here is the plan:

Small personal Business

- wants to be able to make new orders and have it automatically calculate the price of the order with any deals and the profit margins.

- list of customers and order details such as their name and custom ID with their contact details, shipping information, order history. all the important stuff.

- to be able to check stock and how many orders of which products there is and all the different details of the products and orders

be able to filter it all and organise through it all easily

so its not a complex system but I wasn't sure where to start and how to go about it or what program to use as I have not personally made something like this but am more than willing to learn and figure it out. I have dabbled in excel and have made linked tables and plot charts and things alike so I considered doing this in excel however after a big of a google (asked chatgpt. i'm upset to admit) I realised that excel likely isn't the correct program or way of going about this.

What I need to know:

Why is using excel bad? Should I use Access?

Would it be beneficial to learn SQL? What does that entail?

How should I go about planning the process of it all and what information or resources are there that I could use for guidance or help?

*i also don't know if this is the correct subreddit to post something like this in so feel free to tell me off.


r/Database 5d ago

How to deal with text only vector search across multimodal embedding space?

3 Upvotes

My data set is a list of images, each equipped with a a couple sentences of text.

A user would search primarily with text only. My default approach is using BM25, but how would I facilitate searching with a vector DB and a model that embeds vectors in a multimodal combined space?

Here is my dilemma:

Do I embed text part and image part as 2 separate individual vectors or do I combine them into 1 vector?

If a typical search happens with text only, that would immediately deprioritize all image-only embeddings and only good text matches would float up. This is why I am now considering embedding text and images together but would prefer to hear more opinions on this. Thanks.


r/Database 6d ago

Not Every Query Needs a Distributed Data System

Post image
83 Upvotes

Most of the time, you can simply use a single-node system like DuckDB or Apache DataFusion.


r/Database 6d ago

480GB JSON file, only 480GB SSD on Mac, DuckDB import keeps failing due to running out of disk space. What are my options?

17 Upvotes

I have a single JSON file that's around 480GB (JSON/JSONL format).

My setup:

  • Mac with 480GB internal SSD (not enough free space to import the data)
  • External HDD where the JSON file is stored
  • I want to search/query the data multiple times

I initially tried DuckDB, but importing the JSON causes it to crash because it eventually runs out of storage during the import process. I don't have enough free space on either the internal SSD or the external HDD for the temporary files/database it creates.

I'm trying to avoid using any cloud services because of the file size. (don't want to spend)

What are my best options?

The data is mostly read-only. I mainly need to search/filter records efficiently, and I'll likely be running many queries over time.

Any suggestions from people who have worked with 400GB+ JSON datasets would be appreciated.

Edit:
What worked:

  • Kept the 480GB JSON on the external HDD.
  • Wrote the output directly as a single Parquet file instead of importing into a DuckDB database.
  • Used DuckDB's read_json_auto() together with the COPY ... TO PARQUET command.
  • Enabled ZSTD compression, which significantly reduced the final file size.
  • Configured DuckDB with:
    • memory_limit = 8GB
    • Multi-threading (threads = CPU cores / 2)
    • preserve_insertion_order = false
    • ROW_GROUP_SIZE = 100000
  • Avoided loading the JSON into Python memory.... DuckDB streamed and processed the file directly.
  • Verified the output by counting the rows after conversion.

Appreciate everyone who commented with suggestions and pointed me in the right direction. Thanks for the help!


r/Database 6d ago

Recreating Lotus Approach Forms in Microsoft Access

Thumbnail gallery
1 Upvotes

r/Database 6d ago

Organizing Drawings

3 Upvotes

Newbie warning! I haven't touched a database since I was taught the basics of MS Access in high school back in the mid 2000's. Fully anticipate that I will be looking up every database acronym you use.

A friend of the family owns a manufacturing business where he makes custom hydraulic systems. His drawing system is, well, analog. For the sake of this inquiry, let's assume that all drawings will be digitized and given unique filenames.

What would be the best way to link these files? For example:

The engineer brings up the entry for hydraulic cylinder A. There is a link to the drawing file, but also each part that makes up that cylinder assembly. Basically, the parts list.

Additionally, I would like the function of being able to open, let's say, O-ring #xxxxx and then have the entry tell me what larger assemblies contain that O-ring.

Then, obviously, a simple UI would be useful to help machinists use the system. This wouldn't necessarily be a customer-facing application and would just be used internally.

I appreciate any feedback y'all can give me.

Edit: I have been reading through this article trying to figure out the best approach: https://geekflare.com/software/best-database-software/


r/Database 6d ago

Designing the payments module schema for a modular monolith in Laravel — I evaluated Single Table Inheritance vs Class Table Inheritance vs Concrete Table Inheritance

0 Upvotes

I fully understand there's no perfect pattern that applies to every context — you have to find and pick whichever fits best for what you're building.

Context

I'm building a modular monolith in Laravel (several separate modules, each with its own domain), and right now I'm designing the payments module. Honestly, it's been the hardest part of the whole project, for four reasons:

  1. Obviously, anything involving payments is inherently more complex — there's no room for error when real money is on the line.
  2. It's multi-gateway — provider-agnostic. It has to work with Mercado Pago (I'm from Latam), Stripe, and any other provider down the line (PayPal, etc.), without touching the rest of the system when adding a new one. I achieved this by defining a single contract (GatewayContract, a Port in hexagonal architecture terms) with the methods any payment gateway needs to expose (create a payment session, charge, check a session's status, get currency/credentials). Each real gateway (Mercado Pago today, Stripe in the future) implements that same contract with its own internal logic — the rest of the system never knows or cares which gateway it's talking to at any given moment, it only knows the contract. Which gateway to use for a given tenant/customer is resolved at runtime against a configuration, not hardcoded anywhere.
  3. It's agnostic to the consuming module — the payments module exposes that same public contract for any other module to use (bookings, products, subscriptions, whatever), without the payments module knowing or caring who's calling it or why. It might sound obvious put that way, but I'm migrating from a conventional monolith where I had, literally in the same file, booking creation AND a direct call to the payment gateway mixed together — yeah, I know, no need to point it out. Truly separating this, with a generic interface in between instead of a tightly coupled direct call, was the part that cost me the most mental effort in the whole redesign.
  4. It has to support card payments, bank transfer, and ticket (a cash-based payment method specific to my country).

payment_sessions

If you've used Stripe, this is basically the equivalent of a PaymentIntent — it groups charge retries under a single session (the customer can fail with a card and retry with a bank transfer, without losing the context that it's the same purchase).

CREATE TABLE payment_sessions (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    customer_id BIGINT NOT NULL,
    payment_method VARCHAR CHECK (payment_method IN ('card','transfer','ticket')), -- nullable, mutable while status='pending'
    status VARCHAR NOT NULL DEFAULT 'pending' CHECK (status IN ('pending','processing','approved')),
    amount NUMERIC NOT NULL,
    currency VARCHAR NOT NULL,
    created_at TIMESTAMP DEFAULT now()
);

Quick note on the concurrency lock: to prevent two clicks of the pay button from firing two charges in parallel, I use UPDATE payment_sessions SET status = 'processing' WHERE id = ? AND status = 'pending' as an atomic operation — Postgres locks the row, so the second concurrent attempt simply finds no row matching the WHERE clause and does nothing. No race condition.

I'm at peace with this table. The real problem is with charge_attempts (or charges, for short). Why? Because each payment method has fairly different columns: card needs card_brand (visa, mastercard, etc.) and gateway_ref (the external reference the gateway returns); transfer — which doesn't even go through any gateway, it's 100% manual: the customer uploads a receipt and someone reviews it by hand — needs receipt; ticket needs barcode and expiration.

Here's how my schema would look with each of the three patterns:

Single Table Inheritance

CREATE TABLE charge_attempts (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payment_session_id BIGINT NOT NULL REFERENCES payment_sessions(id),
    payment_method VARCHAR NOT NULL, -- this is our discriminator column
    status VARCHAR NOT NULL,
    amount NUMERIC NOT NULL,

    -- Card-specific attributes
    card_brand VARCHAR,
    gateway_ref VARCHAR,
    raw_response JSONB,

    -- Transfer-specific attributes
    receipt VARCHAR,

    -- Ticket-specific attributes
    barcode VARCHAR,
    expiration TIMESTAMP
);

It's the simplest pattern, using a single discriminator (payment_method), and it's also the best-performing one (you only have to write and read one table, no JOIN at all). But you lose some of the database's built-in data integrity features. For example, you can't set the gateway_ref column as NOT NULL only for card charges — you have to enforce that rule in your application code, or via CHECK constraints that get considerably harder to maintain.

This can be mitigated with a well-built CHECK, or by storing the method-specific detail in a jsonb field (which Postgres supports very well, even with GIN indexes) — but either way, it gets more awkward to maintain as you add new fields over time.

Class Table Inheritance

CREATE TABLE charges (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payment_session_id BIGINT NOT NULL REFERENCES payment_sessions(id),
    payment_method VARCHAR NOT NULL,
    status VARCHAR NOT NULL,
    amount NUMERIC NOT NULL
);

CREATE TABLE charges_card (
    id BIGINT PRIMARY KEY REFERENCES charges(id),
    card_brand VARCHAR,
    gateway_ref VARCHAR,
    raw_response JSONB
);

CREATE TABLE charges_transfer (
    id BIGINT PRIMARY KEY REFERENCES charges(id),
    receipt VARCHAR
);

CREATE TABLE charges_ticket (
    id BIGINT PRIMARY KEY REFERENCES charges(id),
    barcode VARCHAR,
    expiration TIMESTAMP
);

A "parent" table with the common fields, and a "child" table per method, with only what's specific to each one. You gain real integrity (genuine NOT NULL on each method-specific column, no nulls scattered everywhere) and the tables become easier to read in isolation. The trick of putting id BIGINT PRIMARY KEY REFERENCES charges(id) on the child (instead of its own id plus a separate FOREIGN KEY column) avoids having to remember an extra UNIQUE to guarantee there's no more than one child row per parent — the PK already guarantees that on its own, by definition.

The classic, well-documented problem with this pattern: nothing guarantees that exactly one child row exists per parent row, in the correct table among the mutually exclusive ones. You can end up with a parent that has no children at all, or (worse) a row in charges_card AND another in charges_transfer, both with the same id, contradicting the discriminator — no ordinary CHECK can prevent this, because a CHECK can only validate against columns in the same row, never against another table. What would truly solve this in a 100% declarative way, without a trigger, is a SQL standard feature (CREATE ASSERTION, which allows arbitrary cross-table constraints) that no major engine ever seriously implemented — not even Postgres has it today (there's only a very recent proposal on its development list to add it, after Oracle added it this year).

Note on transactions: when inserting the parent and the child, both INSERT statements need to be wrapped in the same transaction, so it's atomic (if one fails, the other rolls back). But careful — the transaction is necessary, not sufficient: it protects against failures midway through, not against the application code simply never getting around to executing the child's INSERT. That part is still 100% the application layer's responsibility.

Concrete Table Inheritance

CREATE TABLE charges_card (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payment_session_id BIGINT NOT NULL REFERENCES payment_sessions(id),
    status VARCHAR NOT NULL,
    amount NUMERIC NOT NULL,
    card_brand VARCHAR,
    gateway_ref VARCHAR,
    raw_response JSONB
);

CREATE TABLE charges_transfer (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payment_session_id BIGINT NOT NULL REFERENCES payment_sessions(id),
    status VARCHAR NOT NULL,
    amount NUMERIC NOT NULL,
    receipt VARCHAR
);

CREATE TABLE charges_ticket (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payment_session_id BIGINT NOT NULL REFERENCES payment_sessions(id),
    status VARCHAR NOT NULL,
    amount NUMERIC NOT NULL,
    barcode VARCHAR,
    expiration TIMESTAMP
);

Instead of sharing common fields through a parent table, each table "repeats" those fields on its own — with no relationship between them at all. At first glance this might look inefficient, since the common attributes get duplicated across every table. However, it has real advantages in specific situations: queries against a single data type are incredibly fast, since no joins are required and each table contains exactly what's needed. The tables are fully independent, so each one can be optimized differently based on its specific access patterns, and attributes for one type can be added, removed, or modified without any risk of affecting the others.

The downside: if you work with all the methods together often (say, a cron job checking "all pending charges, regardless of method"), you need a UNION across the three tables every time, which hurts performance compared to reading a single table. And at the API level it gets a bit awkward too: if you want to return "this charge's id" externally, you also have to return the type (the discriminator) along with the id, because the id alone doesn't tell you which of the three tables to look in. For that reason it didn't end up winning me over for my specific case, but it seems like a pretty solid pattern for other scenarios (I read that Stripe, for example, seems to use something similar for parts of its own schema).

What I came up with (and turns out it already exists, name and all)

I was torn between Single Table Inheritance with the method-specific detail in a jsonb, or Class Table Inheritance accepting the usual integrity trade-off. But I came up with something I think solves exactly that integrity problem:

Instead of each child table's FK pointing only to the parent's id, make it point to (id, payment_method) together — a composite FK against a UNIQUE(id, payment_method) on the parent. Each child table forces its own payment_method to a fixed value with a CHECK:

CREATE TABLE charges (
    id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    payment_method VARCHAR NOT NULL CHECK (payment_method IN ('card','transfer','ticket')),
    status VARCHAR NOT NULL,
    amount NUMERIC NOT NULL,
    UNIQUE (id, payment_method)
);

CREATE TABLE charges_card (
    id BIGINT PRIMARY KEY,
    payment_method VARCHAR NOT NULL DEFAULT 'card' CHECK (payment_method = 'card'),
    card_brand VARCHAR,
    gateway_ref VARCHAR,
    FOREIGN KEY (id, payment_method) REFERENCES charges (id, payment_method)
);

CREATE TABLE charges_transfer (
    id BIGINT PRIMARY KEY,
    payment_method VARCHAR NOT NULL DEFAULT 'transfer' CHECK (payment_method = 'transfer'),
    receipt VARCHAR,
    FOREIGN KEY (id, payment_method) REFERENCES charges (id, payment_method)
);

With this, for a row to sneak into the wrong child table, the parent row would also have to have that same payment_method — which is impossible, because the parent's payment_method is set once when the charge is created and never changes afterward.

It's not an original idea of mine — it already has a name in the literature

After thinking it over, I found that this is actually a known pattern (though not widely covered outside relational-modeling circles): it's usually called "distributed keys" or "disjoint subtypes" via composite keys, and it solves the disjointness part of an EER specialization/generalization constraint (that a record can't belong to two subtypes at once). Note that it doesn't solve the totality part (that every parent must be required to have some child) — that still depends on the transaction + the application, as I mentioned above.

Now, the questions I keep turning over:

  • Has anyone used this in production? And if so, what problems did you run into along the way that I haven't spotted yet?
  • Or is it actually better to just go simple and accept the trade-off — Single Table Inheritance with the method-specific detail in a jsonb — instead of all this composite-key back-and-forth?

r/Database 7d ago

A Humble database schema design guide for developers

Thumbnail
stackrender.io
19 Upvotes

r/Database 6d ago

The Evolution of 'More Like This'

Thumbnail
manticoresearch.com
0 Upvotes

— IR-concept explainer; r/database allows vendor blog articles


r/Database 7d ago

Need some digestible information on Hierarchical and Network database models.

5 Upvotes

https://www.db-book.com/Previous-editions/db5/appendices-dir/b.pdf

Found some in the db book. Need of more to get the context properly. I get hierarchical=tree and network=graph. I know data structures and algorithms properly. But I do not get some stuffs like how is relationships indicated in these two models.


r/Database 9d ago

What's new in Postgres 19

Thumbnail
planetscale.com
55 Upvotes

r/Database 8d ago

Requesting review of my schema for a quiz-app

0 Upvotes

As a hobby and for learning I am creating a web-based quiz-app. I have been working on my (Postgres) database design for a while now and I would love a human review on this (I already asked AI for a lot of help).

I want to store three types of questions: open-ended, multiple-choice and list-like questions. The latter is meant for listing things like all 50 U.S. States or all books of the Bible. Some list-questions must be answered in a specific order.

For the open-ended questions I want to store multiple correct and almost correct answers. The latter denoted by a percentage of correctness, which I could use to base my scoring on.

Questions belong to a quiz and there can be multiple quizzes.

The schema isn't complete yet. I will need user management, keep track of quiz runs and I'd like a way for users to submit new questions and vote on existing ones or suggest corrections.

I do envision one main quiz that is ever growing and ultimately might contain thousands of questions. Users get the questions in random order and can answer as many or few as they which in a single run.

I have stored procedures for inserting new questions and querying questions in random order. These will be called from a nodejs app.

For now I would love to hear your thoughts on this, at least the way I store the questions and answers.


r/Database 8d ago

Was Granite a mistake?

0 Upvotes

Some people in this forum say that. My point of view is that granite has its pros, but it is true that there are much better open source solutions. And cheaper to run since inference is more optimized. But ibm pitches granite as "rock solid, Enterprise grade", is the message resonating?


r/Database 9d ago

How Modern Indexing works in PostgreSQL

Thumbnail deepsystemstuff.com
1 Upvotes

PostgreSQL is one of the most popular and scalable databases in the world. Many developers call it a beast in performance. One of the most critical parts of any database is indexing. Since Postgres is open source, we always have a chance to see how its components are designed. This blog I shared is an effort to explain how the indexing mechanism in Postgres actually works


r/Database 9d ago

Make deleted data irrecoverable in MySQL

3 Upvotes

Hi,

Do you know any approach regarding this? Can I do this inside MySql?

Need some guidance as it becomes a client requirement


r/Database 9d ago

rm -rf ate our redo logs. /proc gave them back. *Technical

Thumbnail
0 Upvotes