Compare commits

..

6 Commits

Author SHA1 Message Date
logaritmisk f30ce9d9dc docs(db): note deferred date-format validation (#11) at the Date field arm 2026-06-02 11:12:00 +02:00
logaritmisk 45c1d1b123 test(db): cover authority-kind, cross-vocabulary, localized text, replace/remove, no-op, missing object
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 11:07:43 +02:00
logaritmisk c94fd1638c docs(db): document set_object_fields replace semantics and required-field deferral
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 11:04:45 +02:00
logaritmisk 2b0056c038 feat(db): set_object_fields with registry validation and audited diffs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 10:59:23 +02:00
logaritmisk 2aaf98794f feat(db): add object.fields jsonb column, read it into CatalogueObject
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 10:54:49 +02:00
logaritmisk 7b0f804461 docs: add Plan 5 (Object flexible-field values) implementation plan
object.fields jsonb + set_object_fields with registry validation (type +
term/authority resolution) + audited per-field diffs; typed FieldError.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 10:52:38 +02:00
8 changed files with 1180 additions and 4 deletions
+1
View File
@@ -15,3 +15,4 @@ serde_json.workspace = true
[dev-dependencies] [dev-dependencies]
tokio.workspace = true tokio.workspace = true
time.workspace = true time.workspace = true
serde_json.workspace = true
@@ -0,0 +1,2 @@
-- Flexible field values for a catalogue object, keyed by field-definition key.
ALTER TABLE object ADD COLUMN fields JSONB NOT NULL DEFAULT '{}'::jsonb;
+191 -4
View File
@@ -2,20 +2,20 @@
//! on the caller's connection, so the change and its audit entry commit together. //! on the caller's connection, so the change and its audit entry commit together.
use domain::{ use domain::{
AuditAction, AuditActor, CatalogueObject, FieldChange, NewAuditEvent, ObjectId, ObjectInput, AuditAction, AuditActor, CatalogueObject, FieldChange, FieldType, NewAuditEvent, ObjectId,
Visibility, ObjectInput, Visibility,
}; };
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::Row; use sqlx::Row;
use crate::audit; use crate::{audit, authority, fields, vocab};
/// The entity_type recorded in the audit log for catalogue objects. /// The entity_type recorded in the audit log for catalogue objects.
const ENTITY_TYPE: &str = "object"; const ENTITY_TYPE: &str = "object";
const OBJECT_COLUMNS: &str = "id, object_number, object_name, number_of_objects, \ const OBJECT_COLUMNS: &str = "id, object_number, object_name, number_of_objects, \
brief_description, current_location, current_owner, recorder, recording_date, \ brief_description, current_location, current_owner, recorder, recording_date, \
visibility, created_at, updated_at"; visibility, fields, created_at, updated_at";
/// Create an object and record a `created` audit entry, both on `conn` /// Create an object and record a `created` audit entry, both on `conn`
/// (pass a transaction connection `&mut *tx` so they commit atomically). /// (pass a transaction connection `&mut *tx` so they commit atomically).
@@ -110,6 +110,7 @@ fn map_object(row: sqlx::postgres::PgRow) -> Result<CatalogueObject, sqlx::Error
recorder: row.try_get("recorder")?, recorder: row.try_get("recorder")?,
recording_date: row.try_get("recording_date")?, recording_date: row.try_get("recording_date")?,
visibility, visibility,
fields: row.try_get("fields")?,
created_at: row.try_get("created_at")?, created_at: row.try_get("created_at")?,
updated_at: row.try_get("updated_at")?, updated_at: row.try_get("updated_at")?,
}) })
@@ -259,3 +260,189 @@ pub async fn delete_object(
Ok(true) Ok(true)
} }
/// Why setting flexible field values failed.
#[derive(Debug, thiserror::Error)]
pub enum FieldError {
#[error("object not found")]
ObjectNotFound,
#[error("unknown field: {0}")]
UnknownField(String),
#[error("field `{field}` expects a {expected} value")]
TypeMismatch {
field: String,
expected: &'static str,
},
#[error("field `{field}`: value does not resolve to an existing {kind}")]
Unresolved { field: String, kind: &'static str },
#[error(transparent)]
Db(#[from] sqlx::Error),
}
/// Replace an object's flexible field values, validating each against the registry
/// (type + term/authority resolution), and audit the per-field diff — all on `conn`.
/// A no-op (identical to the current values) writes nothing and records no audit.
///
/// **Replace semantics:** `values` is the *complete* desired set. Omitting a key that
/// was previously set REMOVES it (recorded in the audit as a removal); send every key
/// the caller wants to retain.
///
/// Required-field *completeness* is intentionally NOT enforced here — a caller may set
/// any subset. That check belongs to the publish gate (when moving to
/// `Visibility::Public`, Plan 7).
pub async fn set_object_fields(
conn: &mut sqlx::PgConnection,
actor: AuditActor,
object_id: ObjectId,
values: &serde_json::Map<String, Value>,
) -> Result<(), FieldError> {
let Some(old) = object_by_id(&mut *conn, object_id).await? else {
return Err(FieldError::ObjectNotFound);
};
for (key, value) in values {
validate_field(&mut *conn, key, value).await?;
}
let new_fields = Value::Object(values.clone());
let changes = field_map_changes(&old.fields, &new_fields);
if changes.is_empty() {
return Ok(());
}
sqlx::query("UPDATE object SET fields = $2, updated_at = now() WHERE id = $1")
.bind(object_id.to_uuid())
.bind(&new_fields)
.execute(&mut *conn)
.await?;
audit::record(
&mut *conn,
&NewAuditEvent {
actor,
action: AuditAction::Updated,
entity_type: ENTITY_TYPE.to_owned(),
entity_id: object_id.to_uuid(),
changes,
},
)
.await?;
Ok(())
}
async fn validate_field(
conn: &mut sqlx::PgConnection,
key: &str,
value: &Value,
) -> Result<(), FieldError> {
let def = fields::field_definition_by_key(&mut *conn, key)
.await?
.ok_or_else(|| FieldError::UnknownField(key.to_owned()))?;
match def.field_type {
FieldType::Text => require(value.is_string(), key, "text")?,
FieldType::LocalizedText => require(
value
.as_object()
.is_some_and(|o| o.values().all(Value::is_string)),
key,
"localized-text object {lang: string}",
)?,
FieldType::Integer => require(value.is_i64(), key, "integer")?,
// Format/range validation (real date parsing) is deferred to issue #11;
// here a date field only requires a string value.
FieldType::Date => require(value.is_string(), key, "date string")?,
FieldType::Boolean => require(value.is_boolean(), key, "boolean")?,
FieldType::Term { vocabulary_id } => {
let term_id = parse_uuid(value, key, "term id (uuid string)")?;
if vocab::resolve_term(
&mut *conn,
vocabulary_id,
domain::TermId::from_uuid(term_id),
)
.await?
.is_none()
{
return Err(FieldError::Unresolved {
field: key.to_owned(),
kind: "term",
});
}
}
FieldType::Authority { kind } => {
let authority_id = parse_uuid(value, key, "authority id (uuid string)")?;
match authority::resolve_authority(
&mut *conn,
domain::AuthorityId::from_uuid(authority_id),
)
.await?
{
Some(ref_) if kind.is_none_or(|k| ref_.kind() == k) => {}
_ => {
return Err(FieldError::Unresolved {
field: key.to_owned(),
kind: "authority",
});
}
}
}
}
Ok(())
}
fn require(ok: bool, field: &str, expected: &'static str) -> Result<(), FieldError> {
if ok {
Ok(())
} else {
Err(FieldError::TypeMismatch {
field: field.to_owned(),
expected,
})
}
}
fn parse_uuid(
value: &Value,
field: &str,
expected: &'static str,
) -> Result<uuid::Uuid, FieldError> {
value
.as_str()
.and_then(|s| s.parse::<uuid::Uuid>().ok())
.ok_or_else(|| FieldError::TypeMismatch {
field: field.to_owned(),
expected,
})
}
/// Per-key diff between two flexible-field maps. `before`/`after` are `None` when
/// the key is absent on that side (so adds and removes are captured).
fn field_map_changes(old: &Value, new: &Value) -> Vec<FieldChange> {
let empty = serde_json::Map::new();
let old_map = old.as_object().unwrap_or(&empty);
let new_map = new.as_object().unwrap_or(&empty);
let keys: std::collections::BTreeSet<&String> = old_map.keys().chain(new_map.keys()).collect();
keys.into_iter()
.filter_map(|key| {
let before = old_map.get(key).cloned();
let after = new_map.get(key).cloned();
if before != after {
Some(FieldChange {
field: key.clone(),
before,
after,
})
} else {
None
}
})
.collect()
}
+13
View File
@@ -116,3 +116,16 @@ async fn object_with_date_and_all_none_optionals_round_trips(pool: PgPool) {
.any(|c| c.field == "recording_date") .any(|c| c.field == "recording_date")
); );
} }
#[sqlx::test]
async fn new_object_has_empty_fields(pool: PgPool) {
let db = Db::from_pool(pool);
let mut tx = db.pool().begin().await.unwrap();
let id = catalog::create_object(&mut tx, AuditActor::System, &sample_input("LM-9"))
.await
.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert_eq!(obj.fields, serde_json::json!({}));
}
+13
View File
@@ -71,3 +71,16 @@ async fn migrate_creates_field_definition_tables(pool: PgPool) {
); );
} }
} }
#[sqlx::test]
async fn migrate_adds_object_fields_column(pool: PgPool) {
let db = Db::from_pool(pool);
let exists: Option<bool> = sqlx::query_scalar(
"SELECT true FROM information_schema.columns \
WHERE table_name = 'object' AND column_name = 'fields'",
)
.fetch_optional(db.pool())
.await
.unwrap();
assert_eq!(exists, Some(true));
}
+374
View File
@@ -0,0 +1,374 @@
use db::catalog::FieldError;
use db::{Db, audit, catalog, fields, vocab};
use domain::{
AuditAction, AuditActor, FieldType, LocalizedLabel, NewFieldDefinition, ObjectInput, Visibility,
};
use sqlx::PgPool;
fn obj_input() -> ObjectInput {
ObjectInput {
object_number: "LM-1".into(),
object_name: "vase".into(),
number_of_objects: 1,
brief_description: None,
current_location: None,
current_owner: None,
recorder: None,
recording_date: None,
visibility: Visibility::Draft,
}
}
fn label(text: &str) -> Vec<LocalizedLabel> {
vec![LocalizedLabel {
lang: "en".into(),
label: text.into(),
}]
}
async fn setup_object(db: &Db) -> domain::ObjectId {
let mut tx = db.pool().begin().await.unwrap();
let id = catalog::create_object(&mut tx, AuditActor::System, &obj_input())
.await
.unwrap();
tx.commit().await.unwrap();
id
}
async fn define(db: &Db, key: &str, field_type: FieldType) {
let mut tx = db.pool().begin().await.unwrap();
fields::create_field_definition(
&mut tx,
&NewFieldDefinition {
key: key.into(),
field_type,
required: false,
group_key: None,
labels: label(key),
},
)
.await
.unwrap();
tx.commit().await.unwrap();
}
#[sqlx::test]
async fn sets_scalar_fields_and_audits(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "comments", FieldType::Text).await;
define(&db, "year", FieldType::Integer).await;
define(&db, "on_display", FieldType::Boolean).await;
let values = serde_json::json!({ "comments": "nice", "year": 1850, "on_display": true });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert_eq!(obj.fields["comments"], "nice");
assert_eq!(obj.fields["year"], 1850);
assert_eq!(obj.fields["on_display"], true);
let history = audit::history_for(db.pool(), "object", id.to_uuid())
.await
.unwrap();
assert_eq!(history.last().unwrap().action, AuditAction::Updated);
let changed: Vec<&str> = history
.last()
.unwrap()
.changes
.iter()
.map(|c| c.field.as_str())
.collect();
assert!(
changed.contains(&"comments")
&& changed.contains(&"year")
&& changed.contains(&"on_display")
);
}
#[sqlx::test]
async fn term_field_must_resolve_in_its_vocabulary(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
let material = vocab::create_vocabulary(db.pool(), "material")
.await
.unwrap();
define(
&db,
"material",
FieldType::Term {
vocabulary_id: material.id,
},
)
.await;
let mut tx = db.pool().begin().await.unwrap();
let wood = vocab::add_term(
&mut tx,
&domain::NewTerm {
vocabulary_id: material.id,
external_uri: None,
labels: label("wood"),
},
)
.await
.unwrap();
tx.commit().await.unwrap();
let ok = serde_json::json!({ "material": wood.to_string() });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, ok.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let bad = serde_json::json!({ "material": domain::TermId::new().to_string() });
let mut tx = db.pool().begin().await.unwrap();
let err =
catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await;
assert!(matches!(err, Err(FieldError::Unresolved { .. })));
}
#[sqlx::test]
async fn unknown_field_and_type_mismatch_are_rejected(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "year", FieldType::Integer).await;
let unknown = serde_json::json!({ "nope": "x" });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(
&mut tx,
AuditActor::System,
id,
unknown.as_object().unwrap()
)
.await,
Err(FieldError::UnknownField(_))
));
drop(tx);
let wrong = serde_json::json!({ "year": "not a number" });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, wrong.as_object().unwrap())
.await,
Err(FieldError::TypeMismatch { .. })
));
drop(tx);
}
#[sqlx::test]
async fn authority_field_enforces_kind(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(
&db,
"maker",
FieldType::Authority {
kind: Some(domain::AuthorityKind::Person),
},
)
.await;
let mut tx = db.pool().begin().await.unwrap();
let person = db::authority::create_authority(
&mut tx,
&domain::NewAuthority {
kind: domain::AuthorityKind::Person,
external_uri: None,
labels: label("Carl"),
},
)
.await
.unwrap();
let place = db::authority::create_authority(
&mut tx,
&domain::NewAuthority {
kind: domain::AuthorityKind::Place,
external_uri: None,
labels: label("Stockholm"),
},
)
.await
.unwrap();
tx.commit().await.unwrap();
let ok = serde_json::json!({ "maker": person.to_string() });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, ok.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let bad = serde_json::json!({ "maker": place.to_string() });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await,
Err(FieldError::Unresolved { .. })
));
}
#[sqlx::test]
async fn term_from_wrong_vocabulary_is_rejected(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
let material = vocab::create_vocabulary(db.pool(), "material")
.await
.unwrap();
let technique = vocab::create_vocabulary(db.pool(), "technique")
.await
.unwrap();
define(
&db,
"material",
FieldType::Term {
vocabulary_id: material.id,
},
)
.await;
// a real term, but in the WRONG vocabulary
let mut tx = db.pool().begin().await.unwrap();
let other = vocab::add_term(
&mut tx,
&domain::NewTerm {
vocabulary_id: technique.id,
external_uri: None,
labels: label("forged"),
},
)
.await
.unwrap();
tx.commit().await.unwrap();
let bad = serde_json::json!({ "material": other.to_string() });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await,
Err(FieldError::Unresolved { .. })
));
}
#[sqlx::test]
async fn localized_text_round_trips(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "title", FieldType::LocalizedText).await;
let values = serde_json::json!({ "title": { "sv": "Vas", "en": "Vase" } });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert_eq!(obj.fields["title"]["sv"], "Vas");
assert_eq!(obj.fields["title"]["en"], "Vase");
let bad = serde_json::json!({ "title": { "sv": 5 } });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await,
Err(FieldError::TypeMismatch { .. })
));
}
#[sqlx::test]
async fn replace_semantics_remove_a_field_and_audit_it(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "comments", FieldType::Text).await;
define(&db, "year", FieldType::Integer).await;
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(
&mut tx,
AuditActor::System,
id,
serde_json::json!({ "comments": "x", "year": 1850 })
.as_object()
.unwrap(),
)
.await
.unwrap();
tx.commit().await.unwrap();
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(
&mut tx,
AuditActor::System,
id,
serde_json::json!({ "comments": "x" }).as_object().unwrap(),
)
.await
.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert!(obj.fields.get("year").is_none());
let history = audit::history_for(db.pool(), "object", id.to_uuid())
.await
.unwrap();
let last = history.last().unwrap();
let year = last
.changes
.iter()
.find(|c| c.field == "year")
.expect("year removal recorded");
assert!(year.before.is_some());
assert!(year.after.is_none());
}
#[sqlx::test]
async fn no_op_set_records_no_audit(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "comments", FieldType::Text).await;
let values = serde_json::json!({ "comments": "x" });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let before = audit::history_for(db.pool(), "object", id.to_uuid())
.await
.unwrap()
.len();
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let after = audit::history_for(db.pool(), "object", id.to_uuid())
.await
.unwrap()
.len();
assert_eq!(before, after, "a no-op set must not add an audit entry");
}
#[sqlx::test]
async fn set_on_missing_object_errors(pool: PgPool) {
let db = Db::from_pool(pool);
let mut tx = db.pool().begin().await.unwrap();
let err = catalog::set_object_fields(
&mut tx,
AuditActor::System,
domain::ObjectId::new(),
serde_json::json!({}).as_object().unwrap(),
)
.await;
assert!(matches!(err, Err(FieldError::ObjectNotFound)));
}
+2
View File
@@ -62,6 +62,8 @@ pub struct CatalogueObject {
pub recorder: Option<String>, pub recorder: Option<String>,
pub recording_date: Option<Date>, pub recording_date: Option<Date>,
pub visibility: Visibility, pub visibility: Visibility,
/// Flexible field values (field key -> value), validated against the registry.
pub fields: serde_json::Value,
pub created_at: OffsetDateTime, pub created_at: OffsetDateTime,
pub updated_at: OffsetDateTime, pub updated_at: OffsetDateTime,
} }
@@ -0,0 +1,584 @@
# Object Flexible-Field Values Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Where the registry gets used — a `fields jsonb` column on `object`, with `set_object_fields` that validates each value against the field-definition registry (Plan 4) + resolves term/authority references (Plan 2) + audits the per-field changes (Plan 1/3), all in the write transaction.
**Architecture:** `CatalogueObject` gains a `fields: serde_json::Value` (the raw JSONB map). `db::catalog::set_object_fields` takes the complete desired field map, validates every value (unknown key → error; type mismatch → error; term/authority must resolve), **replaces** the JSONB, and audits the diff against the old map. A typed `FieldError` (using the `db` crate's so-far-unused `thiserror` dep) surfaces validation failures. Required-field *completeness* is NOT enforced here (deferred to the publish gate, Plan 7).
**Tech Stack:** Rust 2024, sqlx 0.8 (`json`), `serde_json` (values + validation), `thiserror` (FieldError). Tests use `#[sqlx::test]`.
## Design decisions (approved)
- Storage: `object.fields jsonb NOT NULL DEFAULT '{}'`, `field_key → value`.
- Value shapes: text→string, localized_text→`{lang: string}`, integer→JSON integer, date→string, boolean→bool, term→term-UUID string (resolves in bound vocab), authority→authority-UUID string (resolves; kind matches if constrained).
- `set_object_fields` = **replace** the whole map, **separate** from `update_object`; audits the diff; no-op skips write+audit (consistent with `update_object`).
- Required-field enforcement deferred to publish (Plan 7).
- Strict per-field rules (date format, min/max, regex) deferred to #11; here `date` validates as a string only.
- Spectrum field-set seeding is a separate follow-on (not this plan).
## Prerequisites
- Postgres for tests with CREATE DATABASE rights; pass `DATABASE_URL` inline. Shell env does not persist between commands. Pass transaction connections as `&mut tx` (NOT `&mut *tx`) to avoid clippy `explicit_auto_deref`.
## File Structure
```
crates/domain/src/object.rs CatalogueObject gains `fields: serde_json::Value`
crates/db/
Cargo.toml serde_json also in [dev-dependencies]
migrations/0005_object_fields.sql
src/catalog.rs + fields in SELECT/map; FieldError; set_object_fields + validation helpers
tests/object_fields.rs
```
---
## Task 1: `fields` column + read it back
**Files:** create `crates/db/migrations/0005_object_fields.sql`; modify `crates/domain/src/object.rs`, `crates/db/src/catalog.rs`, `crates/db/tests/migrate.rs`, `crates/db/Cargo.toml`.
- [ ] **Step 1: Migration.** Create `crates/db/migrations/0005_object_fields.sql`:
```sql
-- Flexible field values for a catalogue object, keyed by field-definition key.
ALTER TABLE object ADD COLUMN fields JSONB NOT NULL DEFAULT '{}'::jsonb;
```
- [ ] **Step 2: Domain.** In `crates/domain/src/object.rs`, add a `fields` field to `CatalogueObject` (after `visibility`, before the timestamps):
```rust
pub visibility: Visibility,
/// Flexible field values (field key -> value), validated against the registry.
pub fields: serde_json::Value,
pub created_at: OffsetDateTime,
pub updated_at: OffsetDateTime,
```
Add `use serde_json` if needed (serde_json is already a domain dependency). `CatalogueObject` derives `Debug, Clone, PartialEq` (NOT `Eq`) — `serde_json::Value` is `PartialEq` but not `Eq`, so do not add `Eq`. `to_input()` is unchanged (it maps only the 9 core mutable fields; `fields` is not part of `ObjectInput`).
- [ ] **Step 3: db reads the column.** In `crates/db/src/catalog.rs`:
- Add `fields` to the `OBJECT_COLUMNS` const (append `, fields`).
- In `map_object`, add: `fields: row.try_get("fields")?,` (between `visibility` and `created_at`). `sqlx` decodes a `jsonb` column directly into `serde_json::Value`.
(`create_object`'s INSERT does NOT list `fields`, so it uses the `'{}'` default — leave the INSERT unchanged.)
- [ ] **Step 4: dev-dep.** In `crates/db/Cargo.toml`, add `serde_json` to `[dev-dependencies]` (it is a normal dep, but integration tests need it in scope):
```toml
[dev-dependencies]
tokio.workspace = true
time.workspace = true
serde_json.workspace = true
```
- [ ] **Step 5: Tests.** Append to `crates/db/tests/migrate.rs`:
```rust
#[sqlx::test]
async fn migrate_adds_object_fields_column(pool: PgPool) {
let db = Db::from_pool(pool);
let exists: Option<bool> = sqlx::query_scalar(
"SELECT true FROM information_schema.columns \
WHERE table_name = 'object' AND column_name = 'fields'",
)
.fetch_optional(db.pool())
.await
.unwrap();
assert_eq!(exists, Some(true));
}
```
And append to `crates/db/tests/catalog.rs`:
```rust
#[sqlx::test]
async fn new_object_has_empty_fields(pool: PgPool) {
let db = Db::from_pool(pool);
let mut tx = db.pool().begin().await.unwrap();
let id = catalog::create_object(&mut tx, AuditActor::System, &sample_input("LM-9")).await.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert_eq!(obj.fields, serde_json::json!({}));
}
```
(`crates/db/tests/catalog.rs` already imports what it needs except possibly `serde_json` — add `use serde_json` if the test uses the `json!` macro and it is not already imported; `serde_json::json!` works fully-qualified as written.)
- [ ] **Step 6: Run + lint.** `DATABASE_URL=<url> cargo test -p db --test migrate --test catalog` and `cargo test -p domain` → all pass. `cargo +nightly fmt`; `DATABASE_URL=<url> cargo clippy -p db -p domain --all-targets -- -D warnings` → clean.
- [ ] **Step 7: Commit.**
```bash
git add crates/domain crates/db
git commit -m "feat(db): add object.fields jsonb column, read it into CatalogueObject"
```
---
## Task 2: `set_object_fields` — validate, write, audit
**Files:** modify `crates/db/src/catalog.rs`; create `crates/db/tests/object_fields.rs`.
- [ ] **Step 1: Write the failing test** `crates/db/tests/object_fields.rs`:
```rust
use db::catalog::FieldError;
use db::{Db, audit, catalog, fields, vocab};
use domain::{
AuditAction, AuditActor, FieldType, LocalizedLabel, NewFieldDefinition, ObjectInput, Visibility,
};
use sqlx::PgPool;
fn obj_input() -> ObjectInput {
ObjectInput {
object_number: "LM-1".into(),
object_name: "vase".into(),
number_of_objects: 1,
brief_description: None,
current_location: None,
current_owner: None,
recorder: None,
recording_date: None,
visibility: Visibility::Draft,
}
}
fn label(text: &str) -> Vec<LocalizedLabel> {
vec![LocalizedLabel { lang: "en".into(), label: text.into() }]
}
async fn setup_object(db: &Db) -> domain::ObjectId {
let mut tx = db.pool().begin().await.unwrap();
let id = catalog::create_object(&mut tx, AuditActor::System, &obj_input()).await.unwrap();
tx.commit().await.unwrap();
id
}
async fn define(db: &Db, key: &str, field_type: FieldType) {
let mut tx = db.pool().begin().await.unwrap();
fields::create_field_definition(
&mut tx,
&NewFieldDefinition { key: key.into(), field_type, required: false, group_key: None, labels: label(key) },
)
.await
.unwrap();
tx.commit().await.unwrap();
}
#[sqlx::test]
async fn sets_scalar_fields_and_audits(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "comments", FieldType::Text).await;
define(&db, "year", FieldType::Integer).await;
define(&db, "on_display", FieldType::Boolean).await;
let values = serde_json::json!({ "comments": "nice", "year": 1850, "on_display": true });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap())
.await
.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert_eq!(obj.fields["comments"], "nice");
assert_eq!(obj.fields["year"], 1850);
assert_eq!(obj.fields["on_display"], true);
let history = audit::history_for(db.pool(), "object", id.to_uuid()).await.unwrap();
// created + the field set
assert_eq!(history.last().unwrap().action, AuditAction::Updated);
let changed: Vec<&str> = history.last().unwrap().changes.iter().map(|c| c.field.as_str()).collect();
assert!(changed.contains(&"comments") && changed.contains(&"year") && changed.contains(&"on_display"));
}
#[sqlx::test]
async fn term_field_must_resolve_in_its_vocabulary(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
let material = vocab::create_vocabulary(db.pool(), "material").await.unwrap();
define(&db, "material", FieldType::Term { vocabulary_id: material.id }).await;
// add a real term
let mut tx = db.pool().begin().await.unwrap();
let wood = vocab::add_term(
&mut tx,
&domain::NewTerm { vocabulary_id: material.id, external_uri: None, labels: label("wood") },
)
.await
.unwrap();
tx.commit().await.unwrap();
// valid term id resolves
let ok = serde_json::json!({ "material": wood.to_string() });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, ok.as_object().unwrap()).await.unwrap();
tx.commit().await.unwrap();
// random uuid does not resolve
let bad = serde_json::json!({ "material": domain::TermId::new().to_string() });
let mut tx = db.pool().begin().await.unwrap();
let err = catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await;
assert!(matches!(err, Err(FieldError::Unresolved { .. })));
}
#[sqlx::test]
async fn unknown_field_and_type_mismatch_are_rejected(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "year", FieldType::Integer).await;
let unknown = serde_json::json!({ "nope": "x" });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, unknown.as_object().unwrap()).await,
Err(FieldError::UnknownField(_))
));
drop(tx);
let wrong = serde_json::json!({ "year": "not a number" });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, wrong.as_object().unwrap()).await,
Err(FieldError::TypeMismatch { .. })
));
}
```
- [ ] **Step 2: Run to verify it fails.** `DATABASE_URL=<url> cargo test -p db --test object_fields` → FAIL.
- [ ] **Step 3: Implement** — add to the top of `crates/db/src/catalog.rs` the `FieldError` type, and append the function + helpers. Add imports as needed (`use crate::{audit, authority, fields, vocab};``audit` is already imported; add `authority, fields, vocab`). Also ensure `use domain::{... TermId, AuthorityId ...}` are available (add to the existing domain import).
`FieldError`:
```rust
/// Why setting flexible field values failed.
#[derive(Debug, thiserror::Error)]
pub enum FieldError {
#[error("object not found")]
ObjectNotFound,
#[error("unknown field: {0}")]
UnknownField(String),
#[error("field `{field}` expects a {expected} value")]
TypeMismatch { field: String, expected: &'static str },
#[error("field `{field}`: value does not resolve to an existing {kind}")]
Unresolved { field: String, kind: &'static str },
#[error(transparent)]
Db(#[from] sqlx::Error),
}
```
`set_object_fields` + helpers:
```rust
/// Replace an object's flexible field values, validating each against the registry
/// (type + term/authority resolution), and audit the per-field diff — all on `conn`.
/// A no-op (identical to the current values) writes nothing and records no audit.
pub async fn set_object_fields(
conn: &mut sqlx::PgConnection,
actor: AuditActor,
object_id: ObjectId,
values: &serde_json::Map<String, Value>,
) -> Result<(), FieldError> {
let Some(old) = object_by_id(&mut *conn, object_id).await? else {
return Err(FieldError::ObjectNotFound);
};
for (key, value) in values {
validate_field(&mut *conn, key, value).await?;
}
let new_fields = Value::Object(values.clone());
let changes = field_map_changes(&old.fields, &new_fields);
if changes.is_empty() {
return Ok(());
}
sqlx::query("UPDATE object SET fields = $2, updated_at = now() WHERE id = $1")
.bind(object_id.to_uuid())
.bind(&new_fields)
.execute(&mut *conn)
.await?;
audit::record(
&mut *conn,
&NewAuditEvent {
actor,
action: AuditAction::Updated,
entity_type: ENTITY_TYPE.to_owned(),
entity_id: object_id.to_uuid(),
changes,
},
)
.await?;
Ok(())
}
async fn validate_field(
conn: &mut sqlx::PgConnection,
key: &str,
value: &Value,
) -> Result<(), FieldError> {
let def = fields::field_definition_by_key(&mut *conn, key)
.await?
.ok_or_else(|| FieldError::UnknownField(key.to_owned()))?;
match def.field_type {
FieldType::Text => require(value.is_string(), key, "text")?,
FieldType::LocalizedText => require(
value.as_object().is_some_and(|o| o.values().all(Value::is_string)),
key,
"localized-text object {lang: string}",
)?,
FieldType::Integer => require(value.is_i64(), key, "integer")?,
FieldType::Date => require(value.is_string(), key, "date string")?,
FieldType::Boolean => require(value.is_boolean(), key, "boolean")?,
FieldType::Term { vocabulary_id } => {
let term_id = parse_uuid(value, key, "term id (uuid string)")?;
if vocab::resolve_term(&mut *conn, vocabulary_id, domain::TermId::from_uuid(term_id))
.await?
.is_none()
{
return Err(FieldError::Unresolved { field: key.to_owned(), kind: "term" });
}
}
FieldType::Authority { kind } => {
let authority_id = parse_uuid(value, key, "authority id (uuid string)")?;
match authority::resolve_authority(&mut *conn, domain::AuthorityId::from_uuid(authority_id)).await? {
Some(r) if kind.is_none_or(|k| r.kind() == k) => {}
_ => return Err(FieldError::Unresolved { field: key.to_owned(), kind: "authority" }),
}
}
}
Ok(())
}
fn require(ok: bool, field: &str, expected: &'static str) -> Result<(), FieldError> {
if ok {
Ok(())
} else {
Err(FieldError::TypeMismatch { field: field.to_owned(), expected })
}
}
fn parse_uuid(value: &Value, field: &str, expected: &'static str) -> Result<uuid::Uuid, FieldError> {
value
.as_str()
.and_then(|s| s.parse::<uuid::Uuid>().ok())
.ok_or_else(|| FieldError::TypeMismatch { field: field.to_owned(), expected })
}
/// Per-key diff between two flexible-field maps. `before`/`after` are `None` when
/// the key is absent on that side (so adds and removes are captured).
fn field_map_changes(old: &Value, new: &Value) -> Vec<FieldChange> {
let empty = serde_json::Map::new();
let old_map = old.as_object().unwrap_or(&empty);
let new_map = new.as_object().unwrap_or(&empty);
let keys: std::collections::BTreeSet<&String> = old_map.keys().chain(new_map.keys()).collect();
keys.into_iter()
.filter_map(|key| {
let before = old_map.get(key).cloned();
let after = new_map.get(key).cloned();
if before != after {
Some(FieldChange { field: key.clone(), before, after })
} else {
None
}
})
.collect()
}
```
Notes:
- `db` already depends on `thiserror`, `uuid`, `serde_json`, `domain`, and has the `vocab`/`authority`/`fields`/`audit` sibling modules — no Cargo changes beyond Task 1's dev-dep.
- `Value` and `FieldChange` are already imported at the top of `catalog.rs` (`use serde_json::{Value, json};` and `use domain::{..., FieldChange, ...}`). If `FieldChange` is not in the existing domain import, add it.
- [ ] **Step 4: Run to verify it passes.** `DATABASE_URL=<url> cargo test -p db --test object_fields` → PASS (3 tests).
- [ ] **Step 5: Lint.** `cargo +nightly fmt`; `DATABASE_URL=<url> cargo clippy -p db --all-targets -- -D warnings` → clean.
- [ ] **Step 6: Commit.**
```bash
git add crates/db
git commit -m "feat(db): set_object_fields with registry validation and audited diffs"
```
---
## Task 3: validation coverage + replace/no-op semantics
**Files:** modify `crates/db/tests/object_fields.rs`.
- [ ] **Step 1: Add tests** to `crates/db/tests/object_fields.rs`:
```rust
#[sqlx::test]
async fn authority_field_enforces_kind(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "maker", FieldType::Authority { kind: Some(domain::AuthorityKind::Person) }).await;
let mut tx = db.pool().begin().await.unwrap();
let person = db::authority::create_authority(
&mut tx,
&domain::NewAuthority {
kind: domain::AuthorityKind::Person,
external_uri: None,
labels: label("Carl"),
},
)
.await
.unwrap();
let place = db::authority::create_authority(
&mut tx,
&domain::NewAuthority {
kind: domain::AuthorityKind::Place,
external_uri: None,
labels: label("Stockholm"),
},
)
.await
.unwrap();
tx.commit().await.unwrap();
// a person resolves
let ok = serde_json::json!({ "maker": person.to_string() });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, ok.as_object().unwrap()).await.unwrap();
tx.commit().await.unwrap();
// a place is the wrong kind
let bad = serde_json::json!({ "maker": place.to_string() });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await,
Err(FieldError::Unresolved { .. })
));
}
#[sqlx::test]
async fn localized_text_round_trips(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "title", FieldType::LocalizedText).await;
let values = serde_json::json!({ "title": { "sv": "Vas", "en": "Vase" } });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap()).await.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert_eq!(obj.fields["title"]["sv"], "Vas");
assert_eq!(obj.fields["title"]["en"], "Vase");
// a non-string member is rejected
let bad = serde_json::json!({ "title": { "sv": 5 } });
let mut tx = db.pool().begin().await.unwrap();
assert!(matches!(
catalog::set_object_fields(&mut tx, AuditActor::System, id, bad.as_object().unwrap()).await,
Err(FieldError::TypeMismatch { .. })
));
}
#[sqlx::test]
async fn replace_semantics_remove_a_field_and_audit_it(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "comments", FieldType::Text).await;
define(&db, "year", FieldType::Integer).await;
// set both
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(
&mut tx,
AuditActor::System,
id,
serde_json::json!({ "comments": "x", "year": 1850 }).as_object().unwrap(),
)
.await
.unwrap();
tx.commit().await.unwrap();
// replace with only `comments` -> `year` removed
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(
&mut tx,
AuditActor::System,
id,
serde_json::json!({ "comments": "x" }).as_object().unwrap(),
)
.await
.unwrap();
tx.commit().await.unwrap();
let obj = catalog::object_by_id(db.pool(), id).await.unwrap().unwrap();
assert!(obj.fields.get("year").is_none());
let history = audit::history_for(db.pool(), "object", id.to_uuid()).await.unwrap();
let last = history.last().unwrap();
let year = last.changes.iter().find(|c| c.field == "year").expect("year removal recorded");
assert!(year.before.is_some());
assert!(year.after.is_none());
}
#[sqlx::test]
async fn no_op_set_records_no_audit(pool: PgPool) {
let db = Db::from_pool(pool);
let id = setup_object(&db).await;
define(&db, "comments", FieldType::Text).await;
let values = serde_json::json!({ "comments": "x" });
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap()).await.unwrap();
tx.commit().await.unwrap();
let before = audit::history_for(db.pool(), "object", id.to_uuid()).await.unwrap().len();
// setting the identical map again is a no-op
let mut tx = db.pool().begin().await.unwrap();
catalog::set_object_fields(&mut tx, AuditActor::System, id, values.as_object().unwrap()).await.unwrap();
tx.commit().await.unwrap();
let after = audit::history_for(db.pool(), "object", id.to_uuid()).await.unwrap().len();
assert_eq!(before, after, "a no-op set must not add an audit entry");
}
#[sqlx::test]
async fn set_on_missing_object_errors(pool: PgPool) {
let db = Db::from_pool(pool);
let mut tx = db.pool().begin().await.unwrap();
let err = catalog::set_object_fields(
&mut tx,
AuditActor::System,
domain::ObjectId::new(),
serde_json::json!({}).as_object().unwrap(),
)
.await;
assert!(matches!(err, Err(FieldError::ObjectNotFound)));
}
```
- [ ] **Step 2: Run.** `DATABASE_URL=<url> cargo test -p db --test object_fields` → PASS (8 tests total).
- [ ] **Step 3: Full workspace check.**
```bash
cargo +nightly fmt --check
DATABASE_URL=<url> cargo clippy --workspace --all-targets -- -D warnings
DATABASE_URL=<url> cargo test --workspace
```
Expected: all green.
- [ ] **Step 4: Commit.**
```bash
git add crates/db
git commit -m "test(db): cover authority-kind, localized text, replace/remove, no-op, missing object"
```
---
## Self-Review (completed)
**Spec coverage (§6.2 values + validation):**
- `fields jsonb` on object; value shapes per type → Tasks 12. ✓
- Validation against registry (type) + term/authority resolution → Task 2 `validate_field`. ✓
- Replace semantics; audited per-field diffs (adds/removes/changes); no-op skip → Task 2 `set_object_fields`/`field_map_changes` + Task 3 tests. ✓
- Required-completeness deferred to publish (Plan 7); strict date/format rules deferred to #11. ✓ (intentional)
- Typed `FieldError` (uses the `db` crate's `thiserror`). ✓
- SQL confined to `db`; `domain` I/O-free (only gains a `serde_json::Value` field). ✓
**Placeholder scan:** none. `<url>` is the documented `DATABASE_URL`.
**Type consistency:** `set_object_fields(&mut PgConnection, AuditActor, ObjectId, &serde_json::Map<String, Value>) -> Result<(), FieldError>` used identically across impl + all tests. `FieldError` variants matched in tests. `validate_field` matches on `FieldType` variants from `domain` (Plan 4); `vocab::resolve_term`/`authority::resolve_authority` signatures (Plan 2) used as defined. `field_map_changes` produces `FieldChange` (Plan 1) consumed by `audit::record`. `CatalogueObject.fields` added in Task 1 is read by `map_object` and asserted in tests.
## Notes for follow-on plans
- **Spectrum field-set seeding** (a follow-on): use `reference/spectrum-5.0-cataloguing-units-of-information.md` to seed `field_definition`s (key + type + vocabulary binding for "use a standard term source" fields, authority binding for "form of name" fields).
- **Required-field completeness** is enforced at the publish gate (Plan 7): when moving to `Visibility::Public`, check all `required` field definitions have a value.
- **Strict per-field validation rules** (date format, ranges, regex) — issue #11.
- The API layer (Plan 7+/10) will call `update_object` (core) and `set_object_fields` (flexible) within one transaction for a single logical edit, yielding two audit entries; consider whether to coalesce.