Functions for using the Photon rendezvous interface. More...

Functions
int	photon_post_recv_buffer_rdma (int proc, void ptr, uint64_t size, int tag, photon_rid request)

int	photon_post_send_buffer_rdma (int proc, void ptr, uint64_t size, int tag, photon_rid request)

int	photon_post_send_request_rdma (int proc, uint64_t size, int tag, photon_rid *request)

int	photon_wait_recv_buffer_rdma (int proc, uint64_t size, int tag, photon_rid *request)

int	photon_wait_send_buffer_rdma (int proc, uint64_t size, int tag, photon_rid *request)

int	photon_wait_send_request_rdma (int tag)

int	photon_post_os_put (photon_rid request, int proc, void *ptr, uint64_t size, int tag, uint64_t r_offset)

int	photon_post_os_get (photon_rid request, int proc, void *ptr, uint64_t size, int tag, uint64_t r_offset)

int	photon_send_FIN (photon_rid request, int proc, int flags)

int	photon_post_os_put_direct (int proc, void ptr, uint64_t size, photonBuffer rbuf, int flags, photon_rid request)

int	photon_post_os_get_direct (int proc, void ptr, uint64_t size, photonBuffer rbuf, int flags, photon_rid request)

int	photon_test (photon_rid request, int flag, int type, photonStatus status)

int	photon_probe_ledger (int proc, int *flag, int type, photonStatus status)

int	photon_wait (photon_rid request)

int	photon_wait_any (int ret_proc, photon_rid ret_req)

int	photon_wait_any_ledger (int ret_proc, photon_rid ret_req)

Detailed Description

Functions for using the Photon rendezvous interface.

The rendezvous interface provides mechanisms to "advertise" buffer regions from either the sender or receiver side that are then targeted by subsequent PUT or GET RDMA operations. This handshake process allows an application to exchange arbitrary user buffers during runtime without managing private buffer data explicitly.

The rendezvous API uses photon_rid "request" handles to to track outstanding operations. Testing for completion either requires a valid photon_rid to check or returns a photon_rid that is completed. Probing is separated by local completion events and ledger completion events. Local events are those operations that have completed on the node itself, e.g., the successful PUT of a buffer that indicates that the source buffer is free to reuse. Ledger events are notifications that a remote sender has written data into the Photon ledger structure and signals that new data is locally available.

A number of configuration thresholds control the backend behavior of this interface. See photon_config_t::eager_buf_size, photon_config_t::small_msg_size, and photon_config_t::ledger_entries.

All functions are non-blocking except where otherwise noted.

Photon rendezvous handshakes

Function Documentation

◆ photon_post_os_get()

int photon_post_os_get	(	photon_rid	request,
		int	proc,
		void *	ptr,
		uint64_t	size,
		int	tag,
		uint64_t	r_offset
	)

Perform a GET operation from the peer proc the remote buffer identified by the previously setup request. The local user buffer that is received into is identified by ptr and size and must have been previously pinned with photon_register_buffer(). The parameter r_offset specifies the GET offset into the remote buffer. A tag is used for matching.

Parameters

request	A rendezvous request previously exchanged
proc	The remote peer to target
ptr	A pointer to the local user buffer to GET into
size	The size of the local buffer in bytes
tag	An integer tag value to match
r_offset	The offset into the remote buffer

Returns: PHOTON_OK on success or an error code

Definition at line 456 of file libphoton.c.

◆ photon_post_os_get_direct()

int photon_post_os_get_direct	(	int	proc,
		void *	ptr,
		uint64_t	size,
		photonBuffer	rbuf,
		int	flags,
		photon_rid *	request
	)

A direct RDMA GET from a remote buffer previously exchanged using the rendezvous interface. This call is similar to the photon_post_os_get() but does not make any assumption about remote completion notification (i.e. FIN), hence this os considered a one-sided, or "direct", RDMA call. The rbuf parameter may be determined by a call to photon_get_buffer_remote() using a previously defined rendezvous request.

Parameters

proc	The remote peer to target
ptr	A pointer to the local user buffer to GET into
size	The size of the local buffer in bytes
rbuf	A photon_buffer_t struct that defines the remote buffer
flags	Flags that control the direct PUT behavior
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 466 of file libphoton.c.

◆ photon_post_os_put()

int photon_post_os_put	(	photon_rid	request,
		int	proc,
		void *	ptr,
		uint64_t	size,
		int	tag,
		uint64_t	r_offset
	)

Perform a PUT operation to the peer proc and the remote buffer identified by the previously setup request. The local user buffer that is communicated is identified by ptr and size and must have been previously pinned with photon_register_buffer(). The parameter r_offset specifies the PUT offset into the remote buffer. A tag is used for matching.

Parameters

request	A rendezvous request previously exchanged
proc	The remote peer to target
ptr	A pointer to the local user buffer to PUT from
size	The size of the local buffer in bytes
tag	An integer tag value to match
r_offset	The offset into the remote buffer

Returns: PHOTON_OK on success or an error code

Definition at line 447 of file libphoton.c.

◆ photon_post_os_put_direct()

int photon_post_os_put_direct	(	int	proc,
		void *	ptr,
		uint64_t	size,
		photonBuffer	rbuf,
		int	flags,
		photon_rid *	request
	)

A direct RDMA PUT to a remote buffer previously exchanged using the rendezvous interface. This call is similar to the photon_post_os_put() but does not make any assumption about remote completion notification (i.e. FIN), hence this is considered a one-sided, or "direct", RDMA call. The rbuf parameter may be determined by a call to photon_get_buffer_remote() using a previously defined rendezvous request.

Parameters

proc	The remote peer to target
ptr	A pointer to the local user buffer to PUT from
size	The size of the local buffer in bytes
rbuf	A photon_buffer_t struct that defines the remote buffer
flags	Flags that control the direct PUT behavior
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 461 of file libphoton.c.

◆ photon_post_recv_buffer_rdma()

int photon_post_recv_buffer_rdma	(	int	proc,
		void *	ptr,
		uint64_t	size,
		int	tag,
		photon_rid *	request
	)

Advertise a receive buffer to the target proc specified by ptr and size. A tag may be used to filter buffer requests via a unique integer identifier. The receive buffer is intended to be the target of a subsequent PUT by the sender at proc.

Parameters

proc	The remote peer to send the buffer request to
ptr	A pointer to the user buffer address to send
size	The size of the buffer in bytes
tag	An integer tag value to filter on
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 409 of file libphoton.c.

◆ photon_post_send_buffer_rdma()

int photon_post_send_buffer_rdma	(	int	proc,
		void *	ptr,
		uint64_t	size,
		int	tag,
		photon_rid *	request
	)

Advertise a send buffer to the target proc specified by ptr and size. A tag may be used to filter buffer requests via a unique integer identifier. The send buffer is intended to be the target of a subsequent GET by the receiver at proc.

Parameters

proc	The remote peer to send the buffer request to
ptr	A pointer to the user buffer address to send
size	The size of the buffer in bytes
tag	An integer tag value to filter on
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 414 of file libphoton.c.

◆ photon_post_send_request_rdma()

int photon_post_send_request_rdma	(	int	proc,
		uint64_t	size,
		int	tag,
		photon_rid *	request
	)

Request a receive buffer of size bytes be made available at proc. A tag may be used to filter buffer requests via a unique integer identifier. The send request is expected to generate a subsequent photon_post_recv_buffer_rdma() call at the proc target back to the calling peer.

Parameters

proc	The remote peer to send the buffer request to
size	The size of the buffer to request in bytes
tag	An integer tag value to filter on
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 419 of file libphoton.c.

◆ photon_probe_ledger()

int photon_probe_ledger	(	int	proc,
		int *	flag,
		int	type,
		photonStatus	status
	)

Probe for the completion of ledger requests outstanding in the Photon library. This is a nonblocking operation that probes the SEND or RECV ledgers for completion entries but does not return, or dequeue, the request(s) itself.

When this call return successfully, flag is set to 1 if a request in the specificed ledger is available, 0 otherwise. If flag is 1, then the status parameter will be filled with information about the pending request.

Parameters

flag	A flag indicating the result of the test
type	The type of ledger event to probe (PHOTON_PROBE_SLEDGER \| PHOTON_PROBE_RLEDGER)
status	A pointer to a photon_status_t struct filled with the request status on success

Returns: PHOTON_OK on success or an error code

Definition at line 496 of file libphoton.c.

◆ photon_send_FIN()

int photon_send_FIN	(	photon_rid	request,
		int	proc,
		int	flags
	)

Inform the peer proc that request has been fulfilled. This method writes a completion entry into the remote peer's ledger. However, the local operation (i.e. PUT/GET) may still be outstanding, which means request should be tested locally after calling this method.

If flags sets PHOTON_REQ_COMPLETED then the library is informed that request is indeed complete without requiring a wait/test for any local completion.

Parameters

request	The request to mark as completed
proc	The remote peer to target
flags	Flags that control the FIN behavior

Returns: PHOTON_OK on success or an error code

Definition at line 471 of file libphoton.c.

◆ photon_test()

int photon_test	(	photon_rid	request,
		int *	flag,
		int *	type,
		photonStatus	status
	)

Test for the completion of a rendezvous request. This is a nonblocking operation that checks the event queue to see if the event associated with the request parameter has completed. It returns: 0 if the event associated with request was in the queue and was successfully poped. 1 if request was not in the request tables. This is not an error if photon_test() is called in a loop and is called multiple times for each request. -1 if an error occured. When photon_test() returns zero (success) the flag parameter has the value: 0 if the event that was poped does not correspond to request, or if none of the operations completed 1 if the event that was poped does correspond to request

When photon_test() returns 0 and flag equals 0 the status structure is also filled

Regardless of the return value and the value of flag, the parameter type will be set to 0 (zero) when the request is of type event and 1 (one) when the request is of type ledger. type is set to 2 when the request was a send/recv

Parameters

request	The request to test
flag	A flag indicating the result of the test
type	The type of the event that was returned on success
status	A pointer to a photon_status_t struct filled with the request status on success

Returns: 0 on success, 1 if request is not found, -1 on error

Definition at line 399 of file libphoton.c.

◆ photon_wait()

int photon_wait ( photon_rid request )

Wait for the completion of the request. This is a blocking call.

Parameters

request The request to wait for

Returns: PHOTON_OK on success or an error code

Definition at line 404 of file libphoton.c.

◆ photon_wait_any()

int photon_wait_any	(	int *	ret_proc,
		photon_rid *	ret_req
	)

Wait for the completion of any local or ledger request outstanding in the Photon library. This is a blocking call.

Parameters

ret_proc	A pointer to an integer that will be filled with the remote peer of the completed request on success
ret_req	A pointer to a photon_rid that will be filled with the completed request on success

Returns: PHOTON_OK on success or an error code

Definition at line 476 of file libphoton.c.

◆ photon_wait_any_ledger()

int photon_wait_any_ledger	(	int *	ret_proc,
		photon_rid *	ret_req
	)

Wait for the receipt of any ledger, and only ledger, request outstanding in the Photon library. This is a blocking call.

Parameters

ret_proc	A pointer to an integer that will be filled with the remote peer of the completed request on success
ret_req	A pointer to a photon_rid that will be filled with the completed request on success

Returns: PHOTON_OK on success or an error code

Definition at line 487 of file libphoton.c.

◆ photon_wait_recv_buffer_rdma()

int photon_wait_recv_buffer_rdma	(	int	proc,
		uint64_t	size,
		int	tag,
		photon_rid *	request
	)

Wait for the receipt of a photon_post_recv_buffer_rdma() call from proc specified by size and tag. The passed in request pointer is returned with a with a unique handle to test for completion.

Parameters

proc	The remote peer to wait on
size	The size of the buffer to match
tag	An integer tag value to match
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 424 of file libphoton.c.

◆ photon_wait_send_buffer_rdma()

int photon_wait_send_buffer_rdma	(	int	proc,
		uint64_t	size,
		int	tag,
		photon_rid *	request
	)

Wait for the receipt of a photon_post_send_buffer_rdma() call from proc specified by size and tag. The passed in request pointer is returned with a with a unique handle to test for completion.

Parameters

proc	The remote peer to wait on
size	The size of the buffer to match
tag	An integer tag value to match
request	A pointer to a photon_rid that is filled in with a unique value on return

Returns: PHOTON_OK on success or an error code

Definition at line 429 of file libphoton.c.

◆ photon_wait_send_request_rdma()

int photon_wait_send_request_rdma ( int tag )

Wait for the receipt of a photon_post_send_buffer_rdma() matching the tag value. This is a blocking call in contrast to the other rendezvous wait methods.

Parameters

tag	An integer tag value to match

Returns: PHOTON_OK on success or an error code

Definition at line 438 of file libphoton.c.