Package org.apache.giraph.worker

Class WorkerContext

  • All Implemented Interfaces:
    org.apache.giraph.aggregators.AggregatorUsage, org.apache.giraph.conf.GiraphConfigurationSettable<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>, org.apache.giraph.conf.ImmutableClassesGiraphConfigurable<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>, org.apache.giraph.worker.WorkerAggregatorUsage, org.apache.giraph.worker.WorkerBroadcastUsage, org.apache.giraph.worker.WorkerGlobalCommUsage, org.apache.giraph.worker.WorkerIndexUsage<org.apache.hadoop.io.WritableComparable>, org.apache.giraph.worker.WorkerReduceUsage, org.apache.hadoop.io.Writable
    Direct Known Subclasses:
    AggregateBenchmark.AggregateBenchmarkWorkerContext, MessageBenchMark.MessageBenchMarkWorkerContext, PageRank.SimplePageRankWorkerContext, SSSP.SSSPWorkerContext, Traverse.TraverseContext
    public abstract class WorkerContext
extends WorkerAggregatorDelegator<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>
implements org.apache.hadoop.io.Writable, org.apache.giraph.worker.WorkerIndexUsage<org.apache.hadoop.io.WritableComparable>
    WorkerContext allows for the execution of user code on a per-worker basis. There's one WorkerContext per worker.

    Giraph worker context is abstract. Our implementation should contains all necessary interfaces needed. see DefaultWorkerContext

    • Constructor Detail

      • WorkerContext

        public WorkerContext()

    • Method Detail

      • setFragment

        public void setFragment​(IFragment fragment)

      • setCurStep

        public void setCurStep​(int step)
        Make sure this function is called after each step.
        Parameters:
        step -

      • incStep

        public void incStep()

      • postApplication

        public abstract void postApplication()
        Finalize the WorkerContext. This method is executed once on each Worker after the last superstep ends.

      • preSuperstep

        public abstract void preSuperstep()
        Execute user code. This method is executed once on each Worker before each superstep starts.

      • getWorkerCount

        public final int getWorkerCount()
        Get number of workers.

        We use fragment fnum to represent fragment number.

        Specified by:
        getWorkerCount in interface org.apache.giraph.worker.WorkerIndexUsage<org.apache.hadoop.io.WritableComparable>
        Returns:
        Number of workers

      • getMyWorkerIndex

        public final int getMyWorkerIndex()
        Get index for this worker
        Specified by:
        getMyWorkerIndex in interface org.apache.giraph.worker.WorkerIndexUsage<org.apache.hadoop.io.WritableComparable>
        Returns:
        Index of this worker

      • getWorkerForVertex

        public final int getWorkerForVertex​(org.apache.hadoop.io.WritableComparable vertexId)
        Specified by:
        getWorkerForVertex in interface org.apache.giraph.worker.WorkerIndexUsage<org.apache.hadoop.io.WritableComparable>

      • getAndClearMessagesFromOtherWorkers

        public List<org.apache.hadoop.io.Writable> getAndClearMessagesFromOtherWorkers()
        Get messages which other workers sent to this worker and clear them (can be called once per superstep)
        Returns:
        Messages received

      • sendMessageToWorker

        public void sendMessageToWorker​(org.apache.hadoop.io.Writable message,
                                int workerIndex)
        Send message to another worker
        Parameters:
        message - Message to send
        workerIndex - Index of the worker to send the message to

      • postSuperstep

        public abstract void postSuperstep()
        Execute user code. This method is executed once on each Worker after each superstep ends.

      • getSuperstep

        public long getSuperstep()
        Retrieves the current superstep.
        Returns:
        Current superstep

      • getTotalNumVertices

        public final long getTotalNumVertices()
        Get the total (all workers) number of vertices that existed in the previous superstep.
        Returns:
        Total number of vertices (-1 if first superstep) (?)

      • getTotalNumEdges

        public final long getTotalNumEdges()
        Get the total (all workers) number of edges that existed in the previous superstep.
        Returns:
        Total number of edges (-1 if first superstep)

      • getContext

        public final org.apache.hadoop.mapreduce.Mapper.Context getContext()
        Get the mapper context
        Returns:
        Mapper context

      • logToCommandLine

        public void logToCommandLine​(String line)
        Call this to log a line to command line of the job. Use in moderation - it's a synchronous call to Job client
        Parameters:
        line - Line to print

      • readFields

        public void readFields​(DataInput dataInput)
                throws IOException
        Specified by:
        readFields in interface org.apache.hadoop.io.Writable
        Throws:
        IOException

      • reduce

        public void reduce​(String name,
                   Object value)
        Reduce value by name.
        Specified by:
        reduce in interface org.apache.giraph.worker.WorkerReduceUsage
        Overrides:
        reduce in class WorkerAggregatorDelegator<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>
        Parameters:
        name - key
        value - value

      • reduceMerge

        public void reduceMerge​(String name,
                        org.apache.hadoop.io.Writable value)
        Specified by:
        reduceMerge in interface org.apache.giraph.worker.WorkerReduceUsage
        Overrides:
        reduceMerge in class WorkerAggregatorDelegator<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>

      • aggregate

        public <A extends org.apache.hadoop.io.Writable> void aggregate​(String name,
                                                                A value)
        Specified by:
        aggregate in interface org.apache.giraph.worker.WorkerAggregatorUsage
        Overrides:
        aggregate in class WorkerAggregatorDelegator<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>

      • getAggregatedValue

        public <A extends org.apache.hadoop.io.Writable> A getAggregatedValue​(String name)
        Specified by:
        getAggregatedValue in interface org.apache.giraph.aggregators.AggregatorUsage
        Overrides:
        getAggregatedValue in class WorkerAggregatorDelegator<org.apache.hadoop.io.WritableComparable,​org.apache.hadoop.io.Writable,​org.apache.hadoop.io.Writable>