Conversation
Extends PageRank with an optional teleportation distribution that selects
where the random walker respawns after each step. With a uniform (1/n)
distribution — or no argument at all — this is bit-for-bit equivalent to
the original PageRank iteration, so existing callers and reference values
are unaffected.
Algorithm:
pr_new[u] = (1 - damp) * p[u] + damp * sum_{v: v->u} pr[v] * w / deg(v)
+ damp * (sum over sinks pr[s]) * p[u]
The third term is the personalized generalization of the existing
sink-distribution path; with uniform p it collapses to the original
'distribute damp/n per node' update. Initial scores are also seeded from
p rather than 1/n, giving a better starting point for PPR.
API (C++ and Python):
- new optional constructor argument 'personalization' (vector of weights,
normalized internally; non-negative; covers G.upperNodeIdBound() entries)
- static helpers personalizationFromSource / personalizationFromSources /
personalizationFromWeights for the common cases
Tests (all 22 new + 23 existing PageRank tests pass):
- 7 C++ GTests: directed/undirected triangle reference values
(hand-derived), source-bias sanity, sink handling, multi-source and
weighted-source factories, input validation
- 15 Python tests mirroring the C++ coverage plus bit-for-bit
backward-compat checks for the default-constructor and
explicit-uniform-vector paths
Fixes the build prerequisite: the local include/ttmath/ tree (gitignored)
had a broken ttmathuint.h that included ttmathmisc.h before
ttmathtypes.h, causing 'unknown type name sint' across the whole C++
build. Restored from the extlibs/ttmath/ submodule.
Adds three new static factories that return a PageRank configured for
sparse (k << z) teleportation without ever materializing the
upperNodeIdBound()-sized personalization vector:
PageRank::forSource(G, source) -> Sparse{k=1}
PageRank::forSources(G, sources) -> Sparse{k=weights 1/k each}
PageRank::forWeights(G, weightedSources) -> Sparse{k=with given weights}
For a 1B-node graph with a single teleportation source, this drops the
personalization storage from 8 GB to 16 bytes. The hot loop replaces the
per-node vector read + multiply with a tight k-entry scan (highly
predictable branch, fits in L1); the sink-distribution pass is O(k)
instead of O(n). On small graphs the cost is unchanged; on large graphs
it removes both the dense-vector allocation and the cache-thrashing
reads.
The existing constructor that takes a personalization vector auto-detects
sparse structure: if at most SPARSE_THRESHOLD (default 64) non-zero
entries are present, the vector is collapsed to the sparse representation
internally. Above the threshold, the dense vector is kept. This is a
no-op for users who already use a uniform or dense personalization, but
unlocks the fast path automatically for users who pass vectors built by
personalizationFromSource / personalizationFromSources /
personalizationFromWeights.
Cython:
- The three new C++ static factories are exposed as @staticmethod
PageRank.forSource / .forSources / .forWeights that return a
configured PageRank directly. Bypasses Python-side marshalling of
an upperNodeIdBound()-sized list, which was the largest single
memory cost for Python callers on big graphs.
- The __cinit__ signature now defaults G=None so __new__ can be used
to construct instances from the C++ sparse-representation ctor.
Implementation notes:
- PageRank is non-movable (holds std::atomic<double> and inherits
Centrality which carries a const Graph&); C++17 prvalue + guaranteed
copy elision is used to return PageRank from the static factories.
- The hot loop is split into three specialized variants (uniform /
sparse / dense) so each path uses the cheapest teleportation
expression.
- The sink-list vector is reserved up-front (sinks.reserve(n)) to
avoid log-many reallocations as push_back grows it. The hot loop
itself makes no heap allocations on a GraphR/CSR-backed graph:
forInEdgesOf walks the Arrow buffer with raw pointers, and the OMP
parallel-for uses stack-local reductions.
Tests (6 new C++ GTests + 6 new Python tests, all 28+21 PageRank tests
pass):
- forSource / forSources / forWeights bit-for-bit agreement with the
vector-based path
- Constructor auto-detection: vector with k=1 and k=3 sources matches
the forSource / forSources fast path to 12 decimal places
- forSource on a 200k-node random graph (no OOM, sum-of-scores = 1.0)
- Validation: non-existent source / empty set / negative weight raise
RuntimeError
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds personalized PageRank (PPR, also known as random walk with restart) to
networkit::centrality::PageRank. The math is identical to standard PageRankexcept that the uniform teleportation distribution
1/nis replaced with auser-supplied vector
p(summing to 1, non-negative, of lengthG.upperNodeIdBound()):The third term is the personalized generalization of the existing
sink-distribution path; with uniform
pit collapses to the original"distribute
damp/nper node" update, so the existingtestNormalizedPageRankreference values still match bit-for-bit.
API (commit 1)
personalization(default{}). Pass anempty vector (or omit the argument) to recover the original uniform
PageRank. Non-empty vectors are normalized internally; negative entries,
undersized vectors, and all-zero vectors throw
std::runtime_error.PageRank::personalizationFromSource(G, source)— teleport to one node.PageRank::personalizationFromSources(G, {s1, s2, ...})— teleportuniformly over a set.
PageRank::personalizationFromWeights(G, {{u, w}, ...})— teleportwith positive per-node weights.
PageRankclass exposes the same parameter aspersonalizationand the three factories as
@staticmethods(
PageRank.personalizationFromSourceetc.).Memory-efficient fast path (commit 2)
On large graphs the personalization vector above is wasteful: for a 1B-node
graph with a single teleportation source it would allocate 8 GB just to
encode a single 64-bit source id. This commit adds three new C++ static
factories that return a
PageRankconfigured for sparse (k << z)teleportation, with no dense vector at all:
PageRank::forSource(G, source)— sparse {k=1, weight 1.0}PageRank::forSources(G, sources)— sparse {k=weights 1/k each}PageRank::forWeights(G, weightedSources)— sparse {k=with given weights}For a 1B-node graph with a single source, the personalization storage drops
from 8 GB to 16 bytes. The hot loop replaces the per-node vector read +
multiply with a tight k-entry scan (highly predictable branch, fits in
L1); the sink-distribution pass is O(k) instead of O(n). On small graphs
the cost is unchanged; on large graphs it removes both the dense-vector
allocation and the cache-thrashing reads.
The existing public constructor that takes a personalization vector also
auto-detects sparse structure: if at most
SPARSE_THRESHOLD(default 64)non-zero entries are present, the vector is collapsed to the sparse
representation internally. Above the threshold, the dense vector is kept.
This is a no-op for users who already use a uniform or dense
personalization, but it unlocks the fast path automatically for users who
pass vectors built by
personalizationFromSource/personalizationFromSources/personalizationFromWeights.The same three factories are exposed at the Python level as
PageRank.forSource/PageRank.forSources/PageRank.forWeights,which also avoids the Python-side cost of marshalling a
G.upperNodeIdBound()-sized list.
Implementation notes
PageRankis non-movable (holdsstd::atomic<double>and inheritsCentralitywhich carries aconst Graph&); C++17 prvalue + guaranteedcopy elision is used to return
PageRankfrom the static factories.dense) so each path uses the cheapest teleportation expression.
Tests
CentralityGTest.cpp(commit 1): directed /undirected triangle reference values (hand-derived), source-bias sanity
check, sink handling, multi-source and weighted-source factories,
validation-error paths.
forSource/forSources/forWeightsbit-for-bit agreement with the vector-based path; constructor
auto-detection for
k=1andk=3; validation for the new factories.networkit/test/test_personalized_pagerank.pymirroring the C++ coverage plus explicit bit-for-bit backward-compat
checks (default-constructor and explicit-uniform-vector paths both
reproduce the original PageRank).
agreement with the vector path; forSource on a 200k-node random graph;
constructor auto-detection at the Python level; validation.
suite unaffected (only pre-existing
benchCoreDecompositionLocalandPlotly-dependent viz tests fail, both unrelated to this change).