Skip to content

Commit 136aca7

Browse files
authored
feat: add IBigSegmentStore interface + Redis and DynamoDB stores (#536)
Ticket: SDK-2363 · Follows #534 ## Summary Adds the public `IBigSegmentStore` interface, the `Membership` / `StoreMetadata` value types, and concrete DynamoDB + Redis implementations. Schema strings match what the Relay Proxy writes. ```cpp auto redis_store = RedisBigSegmentStore::Create("redis://localhost:6379", "prefix"); auto dynamo_store = DynamoDBBigSegmentStore::Create("my-table", "prefix", options); ``` ## Design notes - **`synchronizedOn` parsing.** Stored as DynamoDB N / Redis string. Both stores reject malformed values (non-numeric strings, wrong DynamoDB attribute type) rather than silently returning 0, matching the existing `dynamodb_source.cpp` row validation. - **No hashing here.** The interface contract says callers pass the already-hashed base64 SHA-256 context key; the SDK will hash in the wrapper, not the store implementations. ## Not in scope - `BigSegmentsBuilder` config plumbing. - `BigSegmentStoreWrapper` (LRU cache + staleness polling) and the hashing path. - Evaluator wiring / replacing the `rules.cpp` big-segments TODO. ## Test plan - [x] 7 `Membership` unit tests pass. - [x] 9 `RedisBigSegmentStore` integration tests pass against Redis 7 (docker). - [x] 8 `DynamoDBBigSegmentStore` integration tests pass against DynamoDB Local. - [x] Full server-sdk test suite (468 tests) still green. - [x] Schema constants match what Relay writes (verified against the LocalStack/Redis fixture data). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > New external-store read path for segment targeting with strict schema validation; not yet wired into evaluation, so production flag behavior is unchanged until follow-up PRs land. > > **Overview** > Introduces a **Big Segments persistent store** surface for the server SDK: public `IBigSegmentStore`, inline `Membership` / `StoreMetadata`, and read-only **Redis** and **DynamoDB** backends that follow Relay’s key/schema layout (prefix scoping, include/exclude refs, sync metadata). > > Stores perform point lookups by opaque context hash, build membership via `Membership::FromSegmentRefs` (inclusion wins on overlap), and return errors on malformed DynamoDB attribute types or non-numeric sync timestamps instead of silent empty/zero results. **Config, caching wrapper, hashing, and flag evaluation wiring are not included** in this change. > > Integration tests cover membership/metadata behavior, prefix isolation, and error paths against Redis and DynamoDB Local. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit d2622d9. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent fd0c761 commit 136aca7

14 files changed

Lines changed: 1137 additions & 8 deletions

File tree

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
/** @file dynamodb_big_segment_store.hpp
2+
* @brief Server-Side DynamoDB Big Segments Store
3+
*/
4+
5+
#pragma once
6+
7+
#include <launchdarkly/server_side/integrations/big_segments/ibig_segment_store.hpp>
8+
#include <launchdarkly/server_side/integrations/dynamodb/options.hpp>
9+
10+
#include <tl/expected.hpp>
11+
12+
#include <memory>
13+
#include <string>
14+
15+
namespace Aws::DynamoDB {
16+
class DynamoDBClient;
17+
}
18+
19+
namespace launchdarkly::server_side::integrations {
20+
21+
/**
22+
* @brief DynamoDBBigSegmentStore is a Big Segments persistent store backed by
23+
* Amazon DynamoDB.
24+
*
25+
* Call DynamoDBBigSegmentStore::Create to obtain a new instance, then pass it
26+
* to the SDK via the Big Segments config builder.
27+
*
28+
* The DynamoDB table must already exist and follow the LaunchDarkly schema:
29+
* a String partition key named `namespace` and a String sort key named `key`.
30+
* The same table can be shared with @ref DynamoDBDataSource — Big Segments
31+
* rows occupy their own partition-key values and do not conflict with
32+
* flag/segment rows. The LaunchDarkly Relay Proxy is responsible for
33+
* populating Big Segments data in this table; this class only reads from it.
34+
*
35+
* This implementation is backed by the AWS SDK for C++.
36+
*/
37+
class DynamoDBBigSegmentStore final : public IBigSegmentStore {
38+
public:
39+
/**
40+
* @brief Creates a new DynamoDBBigSegmentStore, or returns an error if
41+
* construction failed.
42+
*
43+
* @param table_name Name of the DynamoDB table to read from. The table
44+
* must already exist; this class does not create it.
45+
*
46+
* @param prefix Optional namespace prefix. When non-empty, Big Segments
47+
* rows live under partition keys `<prefix>:big_segments_user` and
48+
* `<prefix>:big_segments_metadata`. This allows multiple LaunchDarkly
49+
* environments to share a single table.
50+
*
51+
* @param options Optional AWS DynamoDB client configuration. See
52+
* @ref DynamoDBClientOptions. When defaulted, the AWS SDK resolves
53+
* region, endpoint, and credentials from the standard provider chain
54+
* (environment variables, shared config files, instance metadata).
55+
*
56+
* @return A DynamoDBBigSegmentStore, or an error if construction failed.
57+
*/
58+
static tl::expected<std::unique_ptr<DynamoDBBigSegmentStore>, std::string>
59+
Create(std::string table_name,
60+
std::string prefix,
61+
DynamoDBClientOptions options = {});
62+
63+
[[nodiscard]] GetMembershipResult GetMembership(
64+
std::string const& context_hash) const override;
65+
[[nodiscard]] GetMetadataResult GetMetadata() const override;
66+
67+
~DynamoDBBigSegmentStore() override;
68+
69+
private:
70+
DynamoDBBigSegmentStore(
71+
std::unique_ptr<Aws::DynamoDB::DynamoDBClient> client,
72+
std::string table_name,
73+
std::string prefix);
74+
75+
std::unique_ptr<Aws::DynamoDB::DynamoDBClient> client_;
76+
std::string const table_name_;
77+
std::string const prefix_;
78+
std::string const user_namespace_;
79+
std::string const metadata_namespace_;
80+
};
81+
82+
} // namespace launchdarkly::server_side::integrations

libs/server-sdk-dynamodb-source/src/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ target_sources(${LIBNAME}
1414
PRIVATE
1515
${HEADER_LIST}
1616
dynamodb_source.cpp
17+
dynamodb_big_segment_store.cpp
1718
aws_sdk_guard.cpp
1819
client_factory.cpp
1920
)

libs/server-sdk-dynamodb-source/src/dynamodb_attributes.hpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,4 +18,16 @@ inline constexpr char kItemAttribute[] = "item";
1818
// {namespace: "myprefix:$inited", key: "myprefix:$inited"}.
1919
inline constexpr char kInitedNamespace[] = "$inited";
2020

21+
// Big Segments schema. Membership rows use partition key
22+
// "{prefix}:big_segments_user" and sort key {context_hash}, with
23+
// "included" / "excluded" String Set attributes naming segment refs. The
24+
// metadata row uses partition key AND sort key both set to
25+
// "{prefix}:big_segments_metadata", with the sync timestamp stored as a
26+
// Number under "synchronizedOn".
27+
inline constexpr char kBigSegmentsUserNamespace[] = "big_segments_user";
28+
inline constexpr char kBigSegmentsMetadataNamespace[] = "big_segments_metadata";
29+
inline constexpr char kBigSegmentsIncludedAttribute[] = "included";
30+
inline constexpr char kBigSegmentsExcludedAttribute[] = "excluded";
31+
inline constexpr char kBigSegmentsSyncTimeAttribute[] = "synchronizedOn";
32+
2133
} // namespace launchdarkly::server_side::integrations::detail
Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
#include <launchdarkly/server_side/integrations/dynamodb/dynamodb_big_segment_store.hpp>
2+
3+
#include "aws_sdk_guard.hpp"
4+
#include "client_factory.hpp"
5+
#include "dynamodb_attributes.hpp"
6+
#include "prefix.hpp"
7+
8+
#include <aws/core/utils/Outcome.h>
9+
#include <aws/dynamodb/DynamoDBClient.h>
10+
#include <aws/dynamodb/model/AttributeValue.h>
11+
#include <aws/dynamodb/model/GetItemRequest.h>
12+
13+
#include <cerrno>
14+
#include <cstdint>
15+
#include <cstdlib>
16+
#include <exception>
17+
#include <utility>
18+
19+
namespace launchdarkly::server_side::integrations {
20+
21+
namespace {
22+
23+
using detail::kBigSegmentsExcludedAttribute;
24+
using detail::kBigSegmentsIncludedAttribute;
25+
using detail::kBigSegmentsMetadataNamespace;
26+
using detail::kBigSegmentsSyncTimeAttribute;
27+
using detail::kBigSegmentsUserNamespace;
28+
using detail::kPartitionKey;
29+
using detail::kSortKey;
30+
using detail::PrefixedNamespace;
31+
32+
} // namespace
33+
34+
tl::expected<std::unique_ptr<DynamoDBBigSegmentStore>, std::string>
35+
DynamoDBBigSegmentStore::Create(std::string table_name,
36+
std::string prefix,
37+
DynamoDBClientOptions options) {
38+
try {
39+
detail::AwsSdkGuard::Ensure();
40+
auto maybe_client = detail::BuildDynamoDBClient(options);
41+
if (!maybe_client) {
42+
return tl::make_unexpected(std::move(maybe_client.error()));
43+
}
44+
return std::unique_ptr<DynamoDBBigSegmentStore>(
45+
new DynamoDBBigSegmentStore(std::move(*maybe_client),
46+
std::move(table_name),
47+
std::move(prefix)));
48+
} catch (std::exception const& e) {
49+
return tl::make_unexpected(e.what());
50+
}
51+
}
52+
53+
DynamoDBBigSegmentStore::DynamoDBBigSegmentStore(
54+
std::unique_ptr<Aws::DynamoDB::DynamoDBClient> client,
55+
std::string table_name,
56+
std::string prefix)
57+
: client_(std::move(client)),
58+
table_name_(std::move(table_name)),
59+
prefix_(std::move(prefix)),
60+
user_namespace_(PrefixedNamespace(prefix_, kBigSegmentsUserNamespace)),
61+
metadata_namespace_(
62+
PrefixedNamespace(prefix_, kBigSegmentsMetadataNamespace)) {}
63+
64+
DynamoDBBigSegmentStore::~DynamoDBBigSegmentStore() = default;
65+
66+
IBigSegmentStore::GetMembershipResult DynamoDBBigSegmentStore::GetMembership(
67+
std::string const& context_hash) const {
68+
Aws::DynamoDB::Model::GetItemRequest request;
69+
request.SetTableName(table_name_);
70+
request.SetConsistentRead(true);
71+
request.AddKey(kPartitionKey,
72+
Aws::DynamoDB::Model::AttributeValue{user_namespace_});
73+
request.AddKey(kSortKey,
74+
Aws::DynamoDB::Model::AttributeValue{context_hash});
75+
76+
auto outcome = client_->GetItem(request);
77+
if (!outcome.IsSuccess()) {
78+
return tl::make_unexpected(outcome.GetError().GetMessage());
79+
}
80+
81+
auto const& item = outcome.GetResult().GetItem();
82+
83+
std::vector<std::string> included;
84+
std::vector<std::string> excluded;
85+
86+
// GetSS() silently returns an empty vector if the attribute is not
87+
// actually a String Set, so check the type explicitly before reading.
88+
if (auto const it = item.find(kBigSegmentsIncludedAttribute);
89+
it != item.end()) {
90+
if (it->second.GetType() !=
91+
Aws::DynamoDB::Model::ValueType::STRING_SET) {
92+
return tl::make_unexpected(
93+
std::string("DynamoDB Big Segments '") +
94+
kBigSegmentsIncludedAttribute +
95+
"' is not of type STRING_SET");
96+
}
97+
for (auto const& ref : it->second.GetSS()) {
98+
included.emplace_back(ref);
99+
}
100+
}
101+
if (auto const it = item.find(kBigSegmentsExcludedAttribute);
102+
it != item.end()) {
103+
if (it->second.GetType() !=
104+
Aws::DynamoDB::Model::ValueType::STRING_SET) {
105+
return tl::make_unexpected(
106+
std::string("DynamoDB Big Segments '") +
107+
kBigSegmentsExcludedAttribute +
108+
"' is not of type STRING_SET");
109+
}
110+
for (auto const& ref : it->second.GetSS()) {
111+
excluded.emplace_back(ref);
112+
}
113+
}
114+
115+
return Membership::FromSegmentRefs(included, excluded);
116+
}
117+
118+
IBigSegmentStore::GetMetadataResult DynamoDBBigSegmentStore::GetMetadata()
119+
const {
120+
Aws::DynamoDB::Model::GetItemRequest request;
121+
request.SetTableName(table_name_);
122+
request.SetConsistentRead(true);
123+
request.AddKey(kPartitionKey,
124+
Aws::DynamoDB::Model::AttributeValue{metadata_namespace_});
125+
request.AddKey(kSortKey,
126+
Aws::DynamoDB::Model::AttributeValue{metadata_namespace_});
127+
128+
auto outcome = client_->GetItem(request);
129+
if (!outcome.IsSuccess()) {
130+
return tl::make_unexpected(outcome.GetError().GetMessage());
131+
}
132+
133+
auto const& item = outcome.GetResult().GetItem();
134+
if (item.empty()) {
135+
return std::nullopt;
136+
}
137+
138+
auto const it = item.find(kBigSegmentsSyncTimeAttribute);
139+
if (it == item.end()) {
140+
// "absent" sync time is treated as never synchronized rather than
141+
// an error; the wrapper marks the store stale based on the
142+
// resulting nullopt.
143+
return std::nullopt;
144+
}
145+
146+
auto const& raw = it->second.GetN();
147+
if (raw.empty()) {
148+
return tl::make_unexpected(
149+
"DynamoDB Big Segments 'synchronizedOn' is empty or not type N");
150+
}
151+
152+
errno = 0;
153+
char* end = nullptr;
154+
long long const parsed = std::strtoll(raw.c_str(), &end, 10);
155+
if (errno != 0 || end == raw.c_str() || *end != '\0') {
156+
return tl::make_unexpected(
157+
"DynamoDB Big Segments 'synchronizedOn' is not a valid integer");
158+
}
159+
160+
return StoreMetadata{std::chrono::milliseconds{parsed}};
161+
}
162+
163+
} // namespace launchdarkly::server_side::integrations

0 commit comments

Comments
 (0)