Session¶

Session object¶

class graphscope.Session(config: Config | str | None = None, api_client: ApiClient | None = None, **kw)[source]¶

A class for interacting with GraphScope graph computation service cluster.

A Session object encapsulates the environment in which Operation objects are executed/evaluated.

A session may own resources. It is important to release these resources when they are no longer required. To do this, invoke the close() method on the session.

A Session can register itself as default session with as_default(), and all operations after that will use the default session. Session deregister itself as a default session when closed.

The following example demonstrates its usage:

>>> import graphscope as gs

>>> # use session object explicitly
>>> sess = gs.session()
>>> g = sess.g()
>>> pg = g.project(vertices={'v': []}, edges={'e': ['dist']})
>>> r = gs.sssp(g, 4)
>>> sess.close()

>>> # or use a session as default
>>> sess = gs.session().as_default()
>>> g = gs.g()
>>> pg = g.project(vertices={'v': []}, edges={'e': ['dist']})
>>> r = gs.sssp(pg, 4)
>>> sess.close()

We support setup a service cluster and create a RPC session in following ways:

GraphScope graph computation service run in cluster managed by kubernetes.
>>> s = graphscope.session()
Also, Session provides several keyword params for users to define the cluster. You may use the param k8s_gs_image to specify the image for all engine pod, and param k8s_engine_cpu or k8s_engine_mem to specify the resources. More, you can find all params detail in __init__() method.
>>> s = graphscope.session( ... k8s_vineyard_cpu=0.1, ... k8s_vineyard_mem="256Mi", ... vineyard_shared_mem="4Gi", ... k8s_engine_cpu=0.1, ... k8s_engine_mem="256Mi")

or all params can be provided by a json configuration file or configuration dict.

>>> s = graphscope.session(config='/tmp/config.yaml')
>>> # Or
>>> s = graphscope.session(config={'k8s_engine_cpu': 5, 'k8s_engine_mem': '5Gi'})

__enter__()[source]¶: Context management protocol. Returns self and register self as default session.

__exit__(exc_type, exc_value, exc_tb)[source]¶: Unregister self from the default session, close the session and release the resources, ignore all exceptions in close().

__init__(config: Config | str | None = None, api_client: ApiClient | None = None, **kw)[source]¶

Construct a new GraphScope session.

Parameters:

config (dict or str, optional) – The configuration dict or file about how to launch the GraphScope instance. For str, it will identify it as a path and read the configuration file to build a session if file exist. If not specified, the global default configuration will be used Note that it will overwrite explicit parameters. Defaults to None.
api_client – The kube api client used in kubernetes cluster
kw –
Configurable keys. For backward compatibility. For more details, see Config class in config.py addr (str, optional): The endpoint of a pre-launched GraphScope instance with ‘<ip>:<port>’ format.

A new session id will be generated for each session connection.

mode (str, optional): optional values are eager and lazy. Defaults to eager.

Eager execution is a flexible platform for research and experimentation, it provides:
An intuitive interface: Quickly test on small data. Easier debugging: Call ops directly to inspect running models and test changes.

Lazy execution means GraphScope does not process the data till it has to.
It just gathers all the information to a DAG that we feed into it, and processes only when we execute sess.run(fetches)

cluster_type (str, optional): Deploy GraphScope instance on hosts or k8s cluster. Defaults to k8s.
Available options: “k8s” and “hosts”. Note that only support deployed on localhost with hosts mode.

num_workers (int, optional): The number of workers to launch GraphScope engine. Defaults to 2.

preemptive (bool, optional): If True, GraphScope instance will treat resource params
(e.g. k8s_coordinator_cpu) as limits and provide the minimum available value as requests, but this will make pod has a Burstable QOS, which can be preempted by other pods with high QOS. Otherwise, it will set both requests and limits with the same value.

k8s_namespace (str, optional): Contains the namespace to create all resource inside.
If param missing, it will try to read namespace from kubernetes context, or a random namespace will be created and deleted if namespace not exist. Defaults to None.

k8s_service_type (str, optional): Type determines how the GraphScope service is exposed.
Valid options are NodePort, and LoadBalancer. Defaults to NodePort.

k8s_image_registry (str, optional): The GraphScope image registry.

k8s_image_repository (str, optional): The GraphScope image repository.

k8s_image_tag (str, optional): The GraphScope image tag.

k8s_image_pull_policy (str, optional): Kubernetes image pull policy. Defaults to “IfNotPresent”.

k8s_image_pull_secrets (List[str], optional): A list of secret name used to authorize pull image.

k8s_vineyard_image (str, optional): The image of vineyard.

k8s_vineyard_deployment (str, optional): The name of vineyard deployment to use. GraphScope will try to
discovery the deployment from kubernetes cluster, then use it if exists, and fallback to launching a bundled vineyard container otherwise.

k8s_vineyard_cpu (float, optional): Number of CPU cores request for vineyard container. Defaults to 0.2.

k8s_vineyard_mem (str, optional): Number of memory request for vineyard container. Defaults to ‘256Mi’

k8s_engine_cpu (float, optional): Number of CPU cores request for engine container. Defaults to 1.

k8s_engine_mem (str, optional): Number of memory request for engine container. Defaults to ‘4Gi’.

k8s_coordinator_cpu (float, optional): Number of CPU cores request for coordinator. Defaults to 0.5.

k8s_coordinator_mem (str, optional): Number of memory request for coordinator. Defaults to ‘512Mi’.

etcd_addrs (str, optional): The addr of external etcd cluster,
with formats like ‘etcd01:port,etcd02:port,etcd03:port’

k8s_mars_worker_cpu (float, optional):
Minimum number of CPU cores request for Mars worker container. Defaults to 0.2.

k8s_mars_worker_mem (str, optional):
Minimum number of memory request for Mars worker container. Defaults to ‘4Mi’.

k8s_mars_scheduler_cpu (float, optional):
Minimum number of CPU cores request for Mars scheduler container. Defaults to 0.2.

k8s_mars_scheduler_mem (str, optional):
Minimum number of memory request for Mars scheduler container. Defaults to ‘4Mi’.

k8s_coordinator_pod_node_selector (dict, optional):
Node selector to the coordinator pod on k8s. Default is None. See also: https://tinyurl.com/3nx6k7ph

k8s_engine_pod_node_selector = None
Node selector to the engine pod on k8s. Default is None. See also: https://tinyurl.com/3nx6k7ph

with_mars (bool, optional):
Launch graphscope with Mars. Defaults to False.

enabled_engines (str, optional):
Select a subset of engines to enable. Only make sense in k8s mode.

with_dataset (bool, optional):
Create a container and mount aliyun demo dataset bucket to the path /dataset.

k8s_volumes (dict, optional): A dict of k8s volume which represents a directory containing data,
accessible to the containers in a pod. Defaults to {}.

For example, you can mount host path with:

k8s_volumes = {

“my-data”: {
“type”: “hostPath”, “field”: {

”path”: “<path>”, “type”: “Directory”

}, “mounts”: [

{
“mountPath”: “<path1>”

}, {

”mountPath”: “<path2>”

}

]

}

}

Or you can mount PVC with:

k8s_volumes = {

“my-data”: {
“type”: “persistentVolumeClaim”, “field”: {

”claimName”: “your-pvc-name”

}, “mounts”: [

{
“mountPath”: “<path1>”

}

]

}

}

Also, you can mount a single volume with:

k8s_volumes = {

“my-data”: {
“type”: “hostPath”, “field”: {xxx}, “mounts”: {

”mountPath”: “<path1>”

}

}

}

timeout_seconds (int, optional): For waiting service ready (or waiting for delete if
k8s_waiting_for_delete is True).

dangling_timeout_seconds (int, optional): After seconds of client disconnect,
coordinator will kill this graphscope instance. Defaults to 600. Expect this value to be greater than 5 (heartbeat interval). Disable dangling check by setting -1.

k8s_deploy_mode (str, optional): the deploy mode of engines on the kubernetes cluster. Default to eager.
eager: create all engine pods at once lazy: create engine pods when called

k8s_waiting_for_delete (bool, optional): Waiting for service delete or not. Defaults to False.

k8s_client_config (dict, optional):
config_file: Name of the kube-config file. Provide configurable parameters for connecting to remote k8s e.g. “~/.kube/config”

reconnect (bool, optional): When connecting to a pre-launched GraphScope cluster with addr,
the connect request would be rejected with there is still an existing session connected. There are cases where the session still exists and user’s client has lost connection with the backend, e.g., in a jupyter notebook. We have a dangling_timeout_seconds for it, but a more deterministic behavior would be better.

If reconnect is True, the existing session will be reused. It is the user’s responsibility to ensure there’s no such an active client.

Defaults to False.

Raises:

TypeError – If the given argument combination is invalid and cannot be used to create a GraphScope session.

add_lib(resource_name)[source]¶: add the specified resource to the k8s cluster from client machine.

as_default()[source]¶

Obtain a context manager that make this object as default session.

This method is used when a Session is constructed, which will immediately install self as a default session.

Raises:: ValueError – If default session exist in current context.
Returns:: A context manager using this session as the default session.

close()[source]¶

Closes this session.

This method frees all resources associated with the session.

Note that closing will ignore SIGINT and SIGTERM signal and recover later.

connected() → bool[source]¶

Check if the session is still connected and available.

Returns: True or False

property engine_config¶: Show the engine configuration associated with session in json format.

g(incoming_data=None, oid_type='int64', vid_type='uint64', directed=True, generate_eid=True, retain_oid=True, vertex_map='global', compact_edges=False, use_perfect_hash=False) → Graph | GraphDAGNode[source]¶

Construct a GraphScope graph object on the default session.

It will launch and set a session to default when there is no default session found.

See params detail in graphscope.framework.graph.GraphDAGNode

Returns:: Evaluated in eager mode.
Return type:: graphscope.framework.graph.GraphDAGNode

Examples:

>>> import graphscope
>>> g = graphscope.g()

>>> import graphscope
>>> sess = graphscope.session()
>>> g = sess.g() # creating graph on the session "sess"

get_config()[source]¶: Get configuration of the session.

get_vineyard_object_mapping_table()[source]¶: Get the vineyard object mapping table from the old object id to new object id during storing and restoring graph to pvc on the kubernetes cluster.

graphlearn(graph, nodes=None, edges=None, gen_labels=None)[source]¶

Start a graph learning engine.

Parameters:

graph (graphscope.framework.graph.GraphDAGNode) – The graph to create learning instance.
nodes (list, optional) – list of node types that will be used for GNN training, the element of list can be “node_label” or (node_label, features). If the element of the list is a tuple and contains selected feature list, it would use the selected feature list for training. Default is None which use all type of nodes and for the GNN training.
edges (list, optional) – list of edge types that will be used for GNN training. We use (src_label, edge_label, dst_label) to specify one edge type. Default is None which use all type of edges for GNN training.
gen_labels (list, optional) – Alias node and edge labels and extract train/validation/test dataset from original graph for supervised GNN training. The detail is explained in the examples below.

Examples

>>> # Assume the input graph contains one label node `paper` and one edge label `link`.
>>> features = ["weight", "name"] # use properties "weight" and "name" as features
>>> lg = sess.graphlearn(
        graph,
        nodes=[("paper", features)])  # use "paper" node and features for training
        edges=[("paper", "links", "paper")]  # use the `paper->links->papers` edge type for training
        gen_labels=[
            # split "paper" nodes into 100 pieces, and uses random 75 pieces (75%) as training dataset
            ("train", "paper", 100, (0, 75)),
            # split "paper" nodes into 100 pieces, and uses random 10 pieces (10%) as validation dataset
            ("val", "paper", 100, (75, 85)),
            # split "paper" nodes into 100 pieces, and uses random 15 pieces (15%) as test dataset
            ("test", "paper", 100, (85, 100)),
        ]
    )
Note that the training, validation and test datasets are not overlapping. And for unsupervised learning:
>>> lg = sess.graphlearn(
        graph,
        nodes=[("paper", features)])  # use "paper" node and features for training
        edges=[("paper", "links", "paper")]  # use the `paper->links->papers` edge type for training
        gen_labels=[
            # split "paper" nodes into 100 pieces, and uses all pieces as training dataset
            ("train", "paper", 100, (0, 100)),
        ]
    )

property info¶: Show all resource info associated with session in json format.

interactive(graph, params=None, with_cypher=False)[source]¶

Get an interactive engine handler to execute gremlin and cypher queries.

It will return an instance of graphscope.interactive.query.InteractiveQuery,

>>> # close and recreate InteractiveQuery.
>>> interactive_query = sess.interactive(g)
>>> interactive_query.close()
>>> interactive_query = sess.interactive(g)

Parameters:

graph (graphscope.framework.graph.GraphDAGNode) – The graph to create interactive instance.
params – A dict consists of configurations of GIE instance.

Raises:

InvalidArgumentError –

graph is not a property graph.

Returns:

InteractiveQuery to execute gremlin and cypher queries.

Return type:

graphscope.interactive.query.InteractiveQuery

load_from(*args, **kwargs)[source]¶: Load a graph within the session. See more information in graphscope.load_from().

restore_from_pvc(path: str, pvc_name: str)[source]¶

Restores the graphs from the given path in the given PVC. Notice, before calling this function, the KUBECONFIG environment variable should be set to the path of your kubeconfig file.

Parameters:

path – The path in the pv to which the pvc is bound.
pvc_name – The name of the PVC.

Raises:

RuntimeError – If the cluster type is not Kubernetes.

run_fetches(fetches)[source]¶: Run operations of fetches without the session lock.

store_to_pvc(graphIDs, path: str, pvc_name: str)[source]¶

Stores the given graph IDs to the given path with the given PVC. Also, if you want to store graphs of different sessions to the same pv, you’d better to create different pvc for different sessions at first.

Notice, before calling this function, the KUBECONFIG environment variable should be set to the path of your kubeconfig file. And you should make sure that the pvc is bound to the pv and the pv’s capacity is enough to store the graphs.

The method uses the vineyardctl to create a kubernetes job to serialize the selected graphs. For more information, see the vineyardctl documentation.

https://github.com/v6d-io/v6d/tree/main/k8s/cmd#vineyardctl-deploy-backup-job

Parameters:

graph_ids – The list of graph IDs to store. Supported types: - list: list of vineyard.ObjectID or graphscope.Graph
path – The path in the pv to which the pvc is bound.
pvc_name – The name of the PVC.

Raises:

RuntimeError – If the cluster type is not Kubernetes.

Session Functions¶

`graphscope.session`	alias of `Session`
`graphscope.get_default_session`()	Returns the default session for the current context.
`graphscope.has_default_session`()	True if default session exists in current context.
`graphscope.set_option`(**kwargs)	Set the value of specified options.
`graphscope.g`([incoming_data, oid_type, ...])	Construct a GraphScope graph object on the default session.
`graphscope.gremlin`(args, *kwargs)
`graphscope.graphlearn`(graph[, nodes, edges, ...])	Create a graph learning engine.