Skip to content

Commit

Permalink
nvme: export nvme_configure_[sq|cq] to public
Browse files Browse the repository at this point in the history
They have been used when creating admin queue and I/O queues in
nvme_configure_adminq() and nvme_create_io[sq|cq]() to create an
instance of SQ and CQ to be managed in libvfn with ``ctrl->sq[qid]`` and
``ctrl->cq[qid]``.

The problem happens when an application handles admin CQ only when an
interrupt (e.g., MSI-X) comes with its own CQ handling behavior on top
of it.  In this case, nvme_create_iosq() and nvme_create_iocq()
high-level APIs issue create I/O queue admin command by __admin() where
the command is handled in a sync way by reaping the CQ entry right after
the command submission.  It goes to collision of the CQ between
application CQ reaper class and libvfn __admin() callstack.

To solve this issue, export nvme_configure_[sq|cq]() to public to let
application manages admin command submission for creating queues.

Signed-off-by: Minwoo Im <[email protected]>
  • Loading branch information
minwooim committed Jan 14, 2025
1 parent f89e95f commit 1091d29
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 3 deletions.
39 changes: 39 additions & 0 deletions include/vfn/nvme/ctrl.h
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,45 @@ int nvme_del_ctrl(struct nvme_ctrl *ctrl);
*/
int nvme_reset(struct nvme_ctrl *ctrl);

/**
* nvme_configure_sq - Configure a submission queue instance
* @ctrl: Controller to configure a submission queue instance
* @qid: Queue identifier
* @qsize: Queue size
* @cq: Corresponding completion queue instance
* @flags: Submission queue configuration flags
*
* Create a submission queue instance for the given @qid. This allocates
* memory spaces for the submission queue and map it to IOMMU page table for
* the DMA operations.
*
* The newly created SQ instance can be referred by ``ctrl->sq[qid]``.
*
* **Note** This helper does not issue a Create I/O Submission Queue admin
* command to the admin submission queue. Caller should prepare command
* submission with the instance to be configured.
*/
int nvme_configure_sq(struct nvme_ctrl *ctrl, int qid, int qsize,
struct nvme_cq *cq, unsigned long flags);

/**
* nvme_configure_cq - Configure a completion queue instance
* @ctrl: Controller to configure a completion queue instance
* @qid: Queue identifier
* @qsize: Queue size
* @vector: interrupt vector
*
* Create a completion queue instance for the given @qid. This allocates
* memory spaces for the completion queue and map it to IOMMU page table for
* the DMA operations.
*
* The newly created CQ instance can be referred by ``ctrl->cq[qid]``.
*
* **Note** This helper does not issue a Create I/O Completion Queue admin
* command to the admin submission queue. Caller should prepare command
* submission with the instance to be configured.
*/
int nvme_configure_cq(struct nvme_ctrl *ctrl, int qid, int qsize, int vector);

/**
* nvme_configure_adminq - Configure admin sq/cq pair
Expand Down
6 changes: 3 additions & 3 deletions src/nvme/core.c
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ int nvme_del_ctrl(struct nvme_ctrl *ctrl)
return 0;
}

static int nvme_configure_cq(struct nvme_ctrl *ctrl, int qid, int qsize, int vector)
int nvme_configure_cq(struct nvme_ctrl *ctrl, int qid, int qsize, int vector)
{
struct nvme_cq *cq = &ctrl->cq[qid];
uint64_t cap;
Expand Down Expand Up @@ -181,8 +181,8 @@ void nvme_discard_cq(struct nvme_ctrl *ctrl, struct nvme_cq *cq)
memset(cq, 0x0, sizeof(*cq));
}

static int nvme_configure_sq(struct nvme_ctrl *ctrl, int qid, int qsize,
struct nvme_cq *cq, unsigned long UNUSED flags)
int nvme_configure_sq(struct nvme_ctrl *ctrl, int qid, int qsize,
struct nvme_cq *cq, unsigned long UNUSED flags)
{
struct nvme_sq *sq = &ctrl->sq[qid];
uint64_t cap;
Expand Down

0 comments on commit 1091d29

Please sign in to comment.