Analytical Engine API Reference¶

Analytical Engine ¶

class GrapeEngine¶

GrapeEngine is responsible to assemble and start four major components of analytical engine, including VineyardServer, GrapeInstance, Dispatcher and GraphScopeServer.

Public Functions

inline GrapeEngine(std::string host, int port)¶

Construct a new GrapeEngine object in service mode, listening at the host:port for incoming rpc requests.

Parameters:

host –
port –

inline explicit GrapeEngine(std::string dag_file)¶

Construct a new GrapeEngine object in job mode, processing the workflow defined in the dagfile.

Parameters:: dag_file –

inline void Start()¶

inline void Stop()¶

inline int RunDAGFile()¶

class GrapeInstance : public gs::Subscriber¶

MPI management.

This controller initials MPI communication world, assign a rank to each process. According to the assigned rank, determines whether this process runs as a coordinator or a worker. It also in charges of execute commands from coordinator.

Public Functions

explicit GrapeInstance(const grape::CommSpec &comm_spec)¶

void Init(const std::string &vineyard_socket)¶

virtual bl::result<std::shared_ptr<DispatchResult>> OnReceive(std::shared_ptr<CommandDetail> cmd) override¶

class GSObject¶

GSObject is the base class of GraphScope object. Every object which is managed by ObjectManager should inherit GSObject.

Subclassed by gs::AppEntry, gs::IContextWrapper, gs::IFragmentWrapper, gs::Projector, gs::PropertyGraphUtils

Public Functions

inline explicit GSObject(std::string id, ObjectType type)¶

inline virtual ~GSObject()¶

inline const std::string &id() const¶

inline ObjectType type() const¶

inline virtual std::string ToString() const¶

class ObjectManager¶

ObjectManager manages GSObject like fragment wrapper, loaded app and more.

Public Functions

inline bl::result<void> PutObject(std::shared_ptr<GSObject> obj)¶

inline bl::result<void> RemoveObject(const std::string &id)¶

inline bl::result<std::shared_ptr<GSObject>> GetObject(const std::string &id)¶

template<typename T> inline bl::result<std::shared_ptr<T>> GetObject(const std::string &id)¶

inline bool HasObject(const std::string &id)¶

class AnalyticalServer¶

AnalyticalServer is responsible for create and start the gRPC service.

Public Functions

inline AnalyticalServer(std::shared_ptr<Dispatcher> dispatcher, std::string host, int port)¶

void StartServer()¶

void StopServer()¶

struct EngineConfig¶

EngineConfig contains configurations about the analytical engine, such as networkx features in enabled or not, vineyard socket, and vineyard rpc endpoint.

Public Functions

inline std::string ToJsonString() const¶

Public Members

std::string networkx¶

std::string vineyard_socket¶

std::string vineyard_rpc_endpoint¶

std::string enable_java_sdk¶

class Dispatcher¶

The dispatcher broadcast commands to every worker using MPI.

Public Functions

explicit Dispatcher(const grape::CommSpec &comm_spec)¶

void Start()¶

void Stop()¶

std::vector<DispatchResult> Dispatch(std::shared_ptr<CommandDetail> cmd)¶

void Subscribe(std::shared_ptr<Subscriber> subscriber)¶

void SetCommand(std::shared_ptr<CommandDetail> cmd)¶

class DispatchResult¶

DispatchResult wraps computation result, metadata and error message for python client.

Public Types

enum AggregatePolicy¶

Values:

enumerator kPickFirst¶

enumerator kPickFirstNonEmpty¶

enumerator kRequireConsistent¶

enumerator kConcat¶

enumerator kPickFirstNonEmptyGraphDef¶

enumerator kMergeGraphDef¶

Public Functions

DispatchResult() = default¶

inline explicit DispatchResult(int worker_id, rpc::Code error_code = rpc::Code::OK)¶

inline int worker_id() const¶

inline void set_error(rpc::Code error_code, const std::string &message)¶

inline rpc::Code error_code() const¶

inline std::string message()¶

inline bool has_large_data() const¶

inline void set_graph_def(const rpc::graph::GraphDefPb &graph_def, AggregatePolicy policy = AggregatePolicy::kMergeGraphDef)¶

Set the graph metadata. The meta should be kept consistent among all workers.

Parameters:: graph_def –

inline const rpc::graph::GraphDefPb &graph_def() const¶

inline rpc::graph::GraphDefPb &graph_def()¶

inline void set_data(const std::string &data, AggregatePolicy policy = AggregatePolicy::kRequireConsistent, bool large_data = false)¶

inline void set_data(const grape::InArchive &arc, AggregatePolicy policy = AggregatePolicy::kRequireConsistent, bool large_data = false)¶

inline const std::string &data() const¶

inline AggregatePolicy aggregate_policy() const¶

Fragments in GraphScope ¶

template<typename FRAG_T> class FragmentWrapper¶

This is a fragment wrapper, which means a series of methods are provided to serialize/transform the data attached to the fragment. An AddColumn method is provided to add properties to create a new fragment from the original one.

Template Parameters:: FRAG_T –

template<typename OID_T, typename VID_T, typename VDATA_T, typename EDATA_T, typename VERTEX_MAP_T = vineyard::ArrowVertexMap<typename vineyard::InternalType<OID_T>::type, VID_T>, bool COMPACT = false> class ArrowProjectedFragment : public gs::ArrowProjectedFragmentBase, public vineyard::BareRegistered<ArrowProjectedFragment<OID_T, VID_T, VDATA_T, EDATA_T, vineyard::ArrowVertexMap<typename vineyard::InternalType<OID_T>::type, VID_T>, false>>¶

This class represents the fragment projected from ArrowFragment which contains only one vertex label and edge label. The fragment has no label and property.

Template Parameters:

OID_T – OID type
VID_T – VID type
VDATA_T – The type of data attached with the vertex
EDATA_T – The type of data attached with the edge

Public Types

using oid_t = OID_T ¶

using vid_t = VID_T ¶

using internal_oid_t = typename vineyard::InternalType<oid_t>::type¶

using eid_t = vineyard::property_graph_types::EID_TYPE¶

using vertex_range_t = grape::VertexRange<vid_t>¶

using inner_vertices_t = vertex_range_t ¶

using outer_vertices_t = vertex_range_t ¶

using vertices_t = vertex_range_t ¶

using sub_vertices_t = vertex_range_t ¶

using vertex_t = grape::Vertex<vid_t>¶

using nbr_t = arrow_projected_fragment_impl::Nbr<vid_t, eid_t, EDATA_T>¶

using compact_nbr_t = arrow_projected_fragment_impl::CompactNbr<vid_t, eid_t, EDATA_T>¶

using nbr_unit_t = vineyard::property_graph_utils::NbrUnit<vid_t, eid_t>¶

using adj_list_t = arrow_projected_fragment_impl::AdjList<vid_t, eid_t, EDATA_T>¶

using compact_adj_list_t = arrow_projected_fragment_impl::CompactAdjList<vid_t, eid_t, EDATA_T>¶

using const_adj_list_t = adj_list_t ¶

using const_compact_adj_list_t = compact_adj_list_t ¶

using property_vertex_map_t = VERTEX_MAP_T ¶

using vertex_map_t = ArrowProjectedVertexMap<internal_oid_t, vid_t, property_vertex_map_t>¶

using label_id_t = vineyard::property_graph_types::LABEL_ID_TYPE¶

using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE¶

using vdata_t = VDATA_T ¶

using edata_t = EDATA_T ¶

using property_graph_t = vineyard::ArrowFragment<oid_t, vid_t, property_vertex_map_t, COMPACT>¶

using vid_array_t = typename vineyard::ConvertToArrowType<vid_t>::ArrayType¶

using eid_array_t = typename vineyard::ConvertToArrowType<eid_t>::ArrayType¶

template<typename DATA_T> using vertex_array_t = grape::VertexArray<vertices_t, DATA_T>¶

template<typename DATA_T> using inner_vertex_array_t = grape::VertexArray<inner_vertices_t, DATA_T>¶

template<typename DATA_T> using outer_vertex_array_t = grape::VertexArray<outer_vertices_t, DATA_T>¶

Public Functions

inline ~ArrowProjectedFragment()¶

inline void Construct(const vineyard::ObjectMeta &meta) override¶

inline void PrepareToRunApp(const grape::CommSpec &comm_spec, grape::PrepareConf conf)¶

inline fid_t fid() const¶

inline fid_t fnum() const¶

inline label_id_t vertex_label() const¶

inline label_id_t edge_label() const¶

inline prop_id_t vertex_prop_id() const¶

inline prop_id_t edge_prop_id() const¶

inline vertex_range_t Vertices() const¶

inline vertex_range_t InnerVertices() const¶

inline vertex_range_t OuterVertices() const¶

inline vertex_range_t OuterVertices(fid_t fid) const¶

inline const std::vector<vertex_t> &MirrorVertices(fid_t fid) const¶

inline bool GetVertex(const oid_t &oid, vertex_t &v) const¶

inline oid_t GetId(const vertex_t &v) const¶

inline internal_oid_t GetInternalId(const vertex_t &v) const¶

inline fid_t GetFragId(const vertex_t &v) const¶

inline arrow_projected_fragment_impl::TypedArray<VDATA_T>::value_type GetData(const vertex_t &v) const¶

inline bool Gid2Vertex(const vid_t &gid, vertex_t &v) const¶

inline vid_t Vertex2Gid(const vertex_t &v) const¶

inline vid_t GetInnerVerticesNum() const¶

inline vid_t GetOuterVerticesNum() const¶

inline vid_t GetVerticesNum() const¶

inline size_t GetEdgeNum() const¶

inline size_t GetInEdgeNum() const¶

inline size_t GetOutEdgeNum() const¶

inline size_t GetOutgoingEdgeNum() const¶

inline size_t GetIncomingEdgeNum() const¶

inline size_t GetTotalVerticesNum() const¶

inline bool IsInnerVertex(const vertex_t &v) const¶

inline bool IsOuterVertex(const vertex_t &v) const¶

inline bool GetInnerVertex(const oid_t &oid, vertex_t &v) const¶

inline bool GetOuterVertex(const oid_t &oid, vertex_t &v) const¶

inline oid_t GetInnerVertexId(const vertex_t &v) const¶

inline internal_oid_t GetInnerVertexInternalId(const vertex_t &v) const¶

inline oid_t GetOuterVertexId(const vertex_t &v) const¶

inline internal_oid_t GetOuterVertexInternalId(const vertex_t &v) const¶

inline oid_t Gid2Oid(const vid_t &gid) const¶

inline bool Oid2Gid(const oid_t &oid, vid_t &gid) const¶

inline vid_t Oid2Gid(const oid_t &oid) const¶

inline bool InnerVertexGid2Vertex(const vid_t &gid, vertex_t &v) const¶

inline bool OuterVertexGid2Vertex(const vid_t &gid, vertex_t &v) const¶

inline vid_t GetOuterVertexGid(const vertex_t &v) const¶

inline vid_t GetInnerVertexGid(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetIncomingAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<COMPACT_, compact_adj_list_t>::type GetIncomingAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetOutgoingAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<COMPACT_, compact_adj_list_t>::type GetOutgoingAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetIncomingInnerVertexAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetOutgoingInnerVertexAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetIncomingOuterVertexAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetOutgoingOuterVertexAdjList(const vertex_t &v) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetIncomingAdjList(const vertex_t &v, fid_t src_fid) const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, adj_list_t>::type GetOutgoingAdjList(const vertex_t &v, fid_t dst_fid) const¶

inline int GetLocalOutDegree(const vertex_t &v) const¶

inline int GetLocalInDegree(const vertex_t &v) const¶

inline grape::DestList IEDests(const vertex_t &v) const¶

inline size_t IEDestsSize() const¶

inline grape::DestList OEDests(const vertex_t &v) const¶

inline size_t OEDestsSize() const¶

inline grape::DestList IOEDests(const vertex_t &v) const¶

inline size_t IOEDestsSize() const¶

inline std::shared_ptr<vertex_map_t> GetVertexMap()¶

inline const std::shared_ptr<vertex_map_t> GetVertexMap() const¶

inline bool directed() const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, const nbr_unit_t*>::type get_out_edges_ptr() const¶

template<bool COMPACT_ = COMPACT> inline std::enable_if<!COMPACT_, const nbr_unit_t*>::type get_in_edges_ptr() const¶

inline const int64_t *get_oe_offsets_begin_ptr() const¶

inline const int64_t *get_oe_offsets_end_ptr() const¶

inline const int64_t *get_ie_offsets_begin_ptr() const¶

inline const int64_t *get_ie_offsets_end_ptr() const¶

inline arrow_projected_fragment_impl::TypedArray<EDATA_T> &get_edata_array_accessor()¶

inline arrow_projected_fragment_impl::TypedArray<VDATA_T> &get_vdata_array_accessor()¶

inline std::shared_ptr<vineyard::ArrowFragment<oid_t, vid_t, property_vertex_map_t, COMPACT>> get_arrow_fragment()¶

inline vineyard::ObjectID vertex_map_id() const¶

inline bool local_vertex_map() const¶

inline bool compact_edges() const¶

inline bool use_perfect_hash() const¶

Public Static Functions

static inline std::shared_ptr<vineyard::Object> Create()¶

static inline std::shared_ptr<ArrowProjectedFragment<oid_t, vid_t, vdata_t, edata_t, property_vertex_map_t, COMPACT>> Project(std::shared_ptr<vineyard::ArrowFragment<oid_t, vid_t, property_vertex_map_t, COMPACT>> fragment, const label_id_t &v_label, const prop_id_t &v_prop, const label_id_t &e_label, const prop_id_t &e_prop)¶

Public Static Attributes

static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn¶

template<typename OID_T, typename VID_T, typename VERTEX_MAP_T = vineyard::ArrowVertexMap<OID_T, VID_T>> class ArrowProjectedVertexMap : public vineyard::Registered<ArrowProjectedVertexMap<OID_T, VID_T, vineyard::ArrowVertexMap<OID_T, VID_T>>>¶

This class represents the mapping between oid and vid.

Template Parameters:

OID_T – OID type
VID_T – VID type

Public Functions

inline void Construct(const vineyard::ObjectMeta &meta)¶

inline bool GetOid(vid_t gid, oid_t &oid) const¶

inline bool GetGid(fid_t fid, oid_t oid, vid_t &gid) const¶

inline bool GetGid(oid_t oid, vid_t &gid) const¶

inline vid_t Offset2Lid(const vid_t &offset)¶

inline vid_t GetOffsetFromLid(vid_t lid)¶

inline size_t GetTotalVerticesNum() const¶

inline VID_T GetInnerVertexSize(fid_t fid) const¶

inline VID_T GetLidFromGid(vid_t gid) const¶

inline fid_t GetFidFromGid(vid_t gid) const¶

inline bool use_perfect_hash() const¶

Public Static Functions

static inline std::shared_ptr<vineyard::Object> Create()¶

static inline std::shared_ptr<ArrowProjectedVertexMap<OID_T, VID_T, VERTEX_MAP_T>> Project(std::shared_ptr<property_vertex_map_t> vm, label_id_t v_label)¶

template<typename OID_T = vineyard::property_graph_types::OID_TYPE, typename VID_T = vineyard::property_graph_types::VID_TYPE, template<typename OID_T_ = typename vineyard::InternalType<OID_T>::type, typename VID_T_ = VID_T> class VERTEX_MAP_T = vineyard::ArrowVertexMap> class ArrowFragmentLoader : public vineyard::ArrowFragmentLoader<vineyard::property_graph_types::OID_TYPE, vineyard::property_graph_types::VID_TYPE>¶

This loader can load a ArrowFragment from the data source including local file, oss, numpy, pandas and vineyard.

Template Parameters:

OID_T – OID type
VID_T – VID type

Public Functions

inline ArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, const std::vector<std::string> &efiles, const std::vector<std::string> &vfiles, bool directed = true, bool generate_eid = false, bool retain_oid = false, bool compact_edges = false, bool use_perfect_hash = false)¶

inline ArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, std::shared_ptr<detail::Graph> graph_info)¶

~ArrowFragmentLoader() = default¶

inline boost::leaf::result<std::pair<table_vec_t, std::vector<table_vec_t>>> LoadVertexEdgeTables()¶

inline bl::result<table_vec_t> LoadVertexTables()¶

inline bl::result<std::vector<table_vec_t>> LoadEdgeTables()¶

inline boost::leaf::result<vineyard::ObjectID> LoadFragment()¶

inline boost::leaf::result<vineyard::ObjectID> LoadFragmentAsFragmentGroup()¶

inline bl::result<vineyard::ObjectID> AddLabelsToFragment(vineyard::ObjectID frag_id)¶

inline bl::result<vineyard::ObjectID> AddDataToExistedVLabel(vineyard::ObjectID frag_id, label_id_t label_id)¶

inline bl::result<vineyard::ObjectID> AddDataToExistedELabel(vineyard::ObjectID frag_id, label_id_t label_id)¶

inline boost::leaf::result<vineyard::ObjectID> AddLabelsToFragmentAsFragmentGroup(vineyard::ObjectID frag_id)¶

inline bl::result<vineyard::ObjectID> ExtendLabelData(vineyard::ObjectID frag_id, int extend_type)¶

inline bl::result<void> initPartitioner()¶

Warning

doxygenclass: Cannot find class “gs::ArrowFragmentAppender” in doxygen xml output for project “GraphScope” from directory: /home/runner/work/GraphScope/GraphScope/docs/_build/doxygen/xml

Analytical Apps ¶

template<typename FRAG_T, typename CONTEXT_T> class AppBase¶

AppBase is the base class for applications.

It contains an grape::DefaultMessageManager to process messages during execution of application. Apps derived from this class need to process messages explicitly.

Template Parameters:

FRAG_T – Fragment class
CONTEXT_T – App context class

Public Types

using message_manager_t = grape::DefaultMessageManager¶

Public Functions

AppBase() = default¶

virtual ~AppBase() = default¶

virtual void PEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0¶

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters:

graph –
context –
messages –

virtual void IncEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0¶

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters:

graph –
context –
messages –

Public Static Attributes

static constexpr bool need_split_edges = false¶

static constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex¶

static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kOnlyOut¶

template<typename FRAG_T, typename CONTEXT_T> class PropertyAppBase¶

PropertyAppBase is the base class for applications on property graphs.

It contains an PropertyMessageManager to process messages during execution of application. Apps derived from this class need to process messages explicitly.

Template Parameters:

FRAG_T –
CONTEXT_T –

Public Types

using message_manager_t = PropertyMessageManager¶

Public Functions

PropertyAppBase() = default¶

virtual ~PropertyAppBase() = default¶

virtual void PEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0¶

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters:

graph –
context –
messages –

virtual void IncEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0¶

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters:

graph –
context –
messages –

Public Static Attributes

static constexpr bool need_split_edges = false¶

static constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex¶

static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn¶

template<typename FRAG_T, typename CONTEXT_T> class PropertyAutoAppBase¶

PropertyAutoAppBase is the base class for applications on property graph, with an PropertyAutoMessageManager to process messages during execution of application. Apps derived from this class do not need to process messages explicitly.

Template Parameters:

FRAG_T –
CONTEXT_T –

Public Functions

PropertyAutoAppBase() = default¶

virtual ~PropertyAutoAppBase() = default¶

virtual void PEval(const FRAG_T &graph, CONTEXT_T &context) = 0¶

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters:

graph –
context –
messages –

virtual void IncEval(const FRAG_T &graph, CONTEXT_T &context) = 0¶

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters:

graph –
context –
messages –

Public Static Attributes

static constexpr bool need_split_edges = false¶

static constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex¶

static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn¶

template<typename APP_T> class DefaultWorker¶

DefaultWorker manages the computation cycle. DefaultWorker is a kind of serial worker for apps derived from AppBase.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t¶

using context_t = typename APP_T::context_t¶

using message_manager_t = grape::DefaultMessageManager¶

Public Functions

inline DefaultWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)¶

~DefaultWorker() = default¶

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())¶

inline void Finalize()¶

template<class ...Args> inline void Query(Args&&... args)¶

inline std::shared_ptr<context_t> GetContext()¶

inline void Output(std::ostream &os)¶

template<typename APP_T> class ParallelPropertyWorker¶

A parallel worker for labeled fragment.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t¶

using context_t = typename APP_T::context_t¶

using message_manager_t = ParallelPropertyMessageManager¶

Public Functions

inline ParallelPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)¶

virtual ~ParallelPropertyWorker() = default¶

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())¶

inline void Finalize()¶

template<class ...Args> inline void Query(Args&&... args)¶

inline std::shared_ptr<context_t> GetContext()¶

inline void Output(std::ostream &os)¶

template<typename APP_T> class PropertyAutoWorker¶

A Worker manages the computation cycle. PropertyAutoWorker is a kind of worker for apps derived from PropertyAutoAppBase.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t¶

using context_t = typename APP_T::context_t¶

using message_manager_t = PropertyAutoMessageManager<typename APP_T::fragment_t>¶

Public Functions

inline PropertyAutoWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)¶

~PropertyAutoWorker() = default¶

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())¶

inline void Finalize()¶

template<class ...Args> inline void Query(Args&&... args)¶

inline std::shared_ptr<context_t> GetContext()¶

inline void Output(std::ostream &os)¶

class AppEntry : public gs::GSObject ¶

AppEntry is a class manages an application.

AppEntry holds the a group of function pointers to manipulate the AppFrame, such as gs::CreateWorker, DeleteWorker and Query. The method Init must be called to load the library before use.

Public Functions

inline AppEntry(std::string id, std::string lib_path)¶

inline bl::result<void> Init()¶

inline bl::result<std::shared_ptr<void>> CreateWorker(const std::shared_ptr<void> &fragment, const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &spec)¶

inline bl::result<std::shared_ptr<IContextWrapper>> Query(void *worker_handler, const rpc::QueryArgs &query_args, const std::string &context_key, std::shared_ptr<IFragmentWrapper> &frag_wrapper)¶

template<class FunctionType> struct ArgsNum¶

template<typename T> struct ArgsUnpacker¶

ArgsUnpacker is an util to unpack the Any type of proto to C++ datatype.

Template Parameters:: T – The destination type

template<std::size_t index, class FunctionType> struct ArgTypeAt¶

Analytical Workers ¶

template<typename APP_T> class DefaultWorker

DefaultWorker manages the computation cycle. DefaultWorker is a kind of serial worker for apps derived from AppBase.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t

using context_t = typename APP_T::context_t

using message_manager_t = grape::DefaultMessageManager

Public Functions

inline DefaultWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)

~DefaultWorker() = default

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())

inline void Finalize()

template<class ...Args> inline void Query(Args&&... args)

inline std::shared_ptr<context_t> GetContext()

inline void Output(std::ostream &os)

template<typename APP_T> class DefaultPropertyWorker¶

A Worker manages the computation cycle. DefaultPropertyWorker is a kind of worker for apps derived from AppBase.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t¶

using context_t = typename APP_T::context_t¶

using message_manager_t = PropertyMessageManager¶

Public Functions

inline DefaultPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)¶

~DefaultPropertyWorker() = default¶

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())¶

inline void Finalize()¶

template<class ...Args> inline void Query(Args&&... args)¶

inline std::shared_ptr<context_t> GetContext()¶

inline void Output(std::ostream &os)¶

template<typename APP_T> class ParallelPropertyWorker

A parallel worker for labeled fragment.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t

using context_t = typename APP_T::context_t

using message_manager_t = ParallelPropertyMessageManager

Public Functions

inline ParallelPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)

virtual ~ParallelPropertyWorker() = default

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())

inline void Finalize()

template<class ...Args> inline void Query(Args&&... args)

inline std::shared_ptr<context_t> GetContext()

inline void Output(std::ostream &os)

template<typename APP_T> class PropertyAutoWorker

A Worker manages the computation cycle. PropertyAutoWorker is a kind of worker for apps derived from PropertyAutoAppBase.

Template Parameters:: APP_T –

Public Types

using fragment_t = typename APP_T::fragment_t

using context_t = typename APP_T::context_t

using message_manager_t = PropertyAutoMessageManager<typename APP_T::fragment_t>

Public Functions

inline PropertyAutoWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)

~PropertyAutoWorker() = default

inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())

inline void Finalize()

template<class ...Args> inline void Query(Args&&... args)

inline std::shared_ptr<context_t> GetContext()

inline void Output(std::ostream &os)

App’s Context ¶

template<typename CTX_T, typename = void> struct CtxWrapperBuilder¶

This is a utility to build kinds of ContextWrapper instances.

Template Parameters:: CTX_T –

template<typename FRAG_T> class VertexPropertyContext : public ContextBase¶

VertexPropertyContext can hold any number of columns. The context is designed for labeled fragment - ArrowFragment. Compared with LabeledVertexDataContext, the data type and column count can be determined at runtime.

Template Parameters:: FRAG_T – The fragment class (non-labeled fragment only)

Public Types

using fragment_t = FRAG_T ¶

using vertex_t = typename fragment_t::vertex_t¶

using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE¶

using oid_t = typename fragment_t::oid_t¶

Public Functions

inline explicit VertexPropertyContext(const fragment_t &fragment)¶

inline const fragment_t &fragment()¶

inline int64_t add_column(const std::string &name, ContextDataType type)¶

inline std::shared_ptr<IColumn> get_column(int64_t index)¶

inline std::shared_ptr<IColumn> get_column(const std::string &name)¶

template<typename DATA_T> inline std::shared_ptr<Column<fragment_t, DATA_T>> get_typed_column(int64_t index)¶

template<typename DATA_T> inline std::shared_ptr<Column<fragment_t, DATA_T>> get_typed_column(const std::string &name)¶

inline std::vector<std::shared_ptr<IColumn>> &vertex_properties()¶

inline const std::map<std::string, std::shared_ptr<IColumn>> &properties_map()¶

template<typename FRAG_T, typename DATA_T> class VertexDataContextWrapper : public gs::IVertexDataContextWrapper¶

This is the wrapper class for VertexDataContext. A series of methods are provided to transform the data hold by the context.

Template Parameters:

FRAG_T – The fragment class (Non-labeled fragment only)
DATA_T – The Data type hold by context

Public Functions

inline explicit VertexDataContextWrapper(const std::string &id, std::shared_ptr<IFragmentWrapper> frag_wrapper, std::shared_ptr<context_t> ctx)¶

inline virtual std::string context_type() override¶

inline virtual std::shared_ptr<IFragmentWrapper> fragment_wrapper() override¶

inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToNdArray(const grape::CommSpec &comm_spec, const Selector &selector, const std::pair<std::string, std::string> &range) override¶

inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToDataframe(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors, const std::pair<std::string, std::string> &range) override¶

inline virtual bl::result<vineyard::ObjectID> ToVineyardTensor(const grape::CommSpec &comm_spec, vineyard::Client &client, const Selector &selector, const std::pair<std::string, std::string> &range) override¶

inline virtual bl::result<vineyard::ObjectID> ToVineyardDataframe(const grape::CommSpec &comm_spec, vineyard::Client &client, const std::vector<std::pair<std::string, Selector>> &selectors, const std::pair<std::string, std::string> &range) override¶

inline virtual bl::result<std::vector<std::pair<std::string, std::shared_ptr<arrow::Array>>>> ToArrowArrays(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors) override¶

template<typename FRAG_T, typename DATA_T> class TensorContext : public ContextBase¶

TensorContext is designed for holding a bunch of computation results. The TensorContext should be used if the number of elements are not related to the number of the vertex.

Template Parameters:

FRAG_T –
DATA_T –

Public Types

using data_t = DATA_T ¶

Public Functions

inline explicit TensorContext(const fragment_t &fragment)¶

inline const fragment_t &fragment()¶

inline void assign(const std::vector<data_t> &data, const std::vector<size_t> &shape)¶

inline void assign(const data_t &data)¶

inline void set_shape(std::vector<std::size_t> shape)¶

inline std::vector<size_t> shape() const¶

inline tensor_t &tensor()¶

template<typename FRAG_T, typename DATA_T, typename = void> class TensorContextWrapper : public gs::ITensorContextWrapper¶

TensorContextWrapper is the wrapper class for TensorContext.

Template Parameters:

FRAG_T –
DATA_T –

Public Functions

inline explicit TensorContextWrapper(const std::string &id, std::shared_ptr<IFragmentWrapper> frag_wrapper, std::shared_ptr<context_t> ctx)¶

inline virtual std::string context_type() override¶

inline virtual std::shared_ptr<IFragmentWrapper> fragment_wrapper() override¶

inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToNdArray(const grape::CommSpec &comm_spec, uint32_t axis) override¶

inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToDataframe(const grape::CommSpec &comm_spec) override¶

inline virtual bl::result<vineyard::ObjectID> ToVineyardTensor(const grape::CommSpec &comm_spec, vineyard::Client &client, uint32_t axis) override¶

inline virtual bl::result<vineyard::ObjectID> ToVineyardDataframe(const grape::CommSpec &comm_spec, vineyard::Client &client) override¶

inline virtual bl::result<std::vector<std::pair<std::string, std::shared_ptr<arrow::Array>>>> ToArrowArrays(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors) override¶

Analytical Engine API Reference¶