Question about the loading of Friendster

[TracyRixner]

@ivanium @kevalvora @josehu07
Hi authors,
Good work!
I use the c5n.4xlarge instance. However, I find the command to prepare dataset

./prepare <PathToRawGraph> <Undirected? (0/1)> <NumVertices> <NumPartitions> <PathToRawFeatures> <DimFeatures> <PathToRawLabels> <LabelKinds>

There is an Out of Memory error.(42GB is not enough) To run friendster, do I need to use a large memory node to make the partition first? And then distribute those partition to each node, right? How large should the RAM be?

Thanks for the help!

[ivanium]

Hi Tracy,

Thank you for your interest in Dorylus.

Yes, you are right that partitioning large graphs can take a huge amount of memory. I didn't have concrete memory consumption numbers right now, but I would suggest to use AWS r-series instances, specifically I recommend at least 256GB memory for friendster. For example, r6i.8xlarge is a good choice.

Hope this helps, and good luck!