Bphilip/add mofed and dependencies #11479

binujp · 2024-12-17T02:06:11Z

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

Summary

What does the PR accomplish, why was it needed?

This PR brings in MOFED and dependent packages required to build our AI/ML distribution story. Immediately HPC images require infiniband for GPU-clustering. MOFED is thus a core requirement. There are first and third party requirments for MOFED drivers directly and dependent packages.

CUDA and GdrCopy drivers need MOFED as a build-time and runtime requirements. This is not satisfied today and CUDA driver is built without MOFED support. These packages getting integrated into core will enable that work stream to be unblocked as well.

Change Log

There are 18 new packages being added by this change to the SPECS directory. There are 8 new SIGNED-SPECS being added too. They are either directly required to build out MOFED and support infrastructure or are tools required to provide a complete infiniband solution like tools to configure the hw/driver and debug issues.

fwctl
ibarr
ibsim
iser
isert
knem
mft_kernel
mlnx-ethtool
mlnx-iproute2
mlnx-nfsrdma
mlnx-ofa_kernel
mlx-steering-dump
multiperf rshim
sockperf
srp
xpmem-lib
xpmem

Does this affect the toolchain?

No

YES/NO

Associated issues

#xxxx

Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-YYYY-XXXX

Test Methodology

Pipeline build id: xxxx
https://dev.azure.com/mariner-org/mariner/_build/results?buildId=709280&view=results

The RPMs from a local build of these specs were installed on A100, V100 and T4 GPU based VMs and functionality verified.

Infiniband is a requirement for AI/ML deployments for inter-GPU communication and scale out GPU clusters. HPC team built HPC azure linux image needs infiniband mofed drivers. Their major customer singularity is on a deadline to move from Ubuntu to azure linux for AI/ML workloads. All of the required sources are open source and have spec's which are already being used by NVIDIA. This PR brings to Azure Linux MOFED driver for infiniband driver and all dependencies to help use, manage and debug the stack. These modules have already been built, integrated into an HPC image and tested on an existing 16 mode cluster owned by HPC team. Performance characteristics are within specified tolerance limits.

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

SPECS/ibsim/ibsim.spec

PawelWMS · 2025-01-13T21:09:51Z

SPECS/iser/iser.spec

+# Copyright (c) 2014 Mellanox Technologies. All rights reserved.
+#
+# This Software is licensed under one of the following licenses:
+#
+# 1) under the terms of the "Common Public License 1.0" a copy of which is
+#    available from the Open Source Initiative, see
+#    http://www.opensource.org/licenses/cpl.php.
+#
+# 2) under the terms of the "The BSD License" a copy of which is
+#    available from the Open Source Initiative, see
+#    http://www.opensource.org/licenses/bsd-license.php.
+#
+# 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+#    copy of which is available from the Open Source Initiative, see
+#    http://www.opensource.org/licenses/gpl-license.php.
+#
+# Licensee has the right to choose one of the above licenses.
+#
+# Redistributions of source code must retain the above copyright
+# notice and one of the license notices.
+#
+# Redistributions in binary form must reproduce both the above copyright
+# notice, one of the license notices in the documentation


I'm not sure how to interpret this licensing comment and the source attributions (we mention both Nvidia and claim we've done this ourselves at the same time). I think we need more clarify on the licensing of the new specs.

I took this line

Licensee has the right to choose one of the above licenses.

as the guidance. Does that work?

Added license copyright files as needed for SPECS fwctl, iser, mofed kernel, srp etc.

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

SPECS/fwctl/fwctl.spec

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

SPECS-SIGNED/iser-signed/iser-signed.spec

SPECS-SIGNED/isert-signed/isert-signed.spec

SPECS-SIGNED/knem-signed/knem-modules-signed.spec

PawelWMS · 2025-01-15T21:01:21Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+%{!?_name: %define _name fwctl}
+%{!?_version: %define _version 24.10}
+%{!?_release: %define _release OFED.24.10.0.6.7.1}


Why define these _version separately? Won't the %{version} work? Similar question for _name. And what's the logic behind _release? I see it trickles down to _kmp_rel for instance but I don't see _kmp_rel used anywhere?

This version is defined from MOFED SPECS for alignment. Hence keeping it same for furture updates.

PawelWMS · 2025-01-15T21:02:24Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+%endif
+%global _kmp_rel %{_release1}%{?_kmp_build_num}%{?_dist}
+
+Summary:	 %{_name} Driver


With the specs done, please also update the version_release_matching_groups inside toolkit/scripts/check_entangled_specs.py. This is part of the PR check, which makes sure "entangled" specs get updated in lockstep within one PR. In case of (regular, signed) spec pairs, if you only update the regular one and forget about this one here, we'd end up either publishing a newer version without the signed .ko files or not publishing the new version at all (I'd need to check the details).

Sign pipeline changes are coming from another PR. Is this can be done with signing pipeline changes for signed SPECs?

PawelWMS · 2025-01-15T23:10:48Z

toolkit/scripts/spec_source_attributions.py

@@ -17,6 +17,7 @@
    "Fedora (ISC)":                   r'\n-\s+Initial (CBL-Mariner|Azure Linux) import from Fedora \d+ \(license: ISC\)(\.|\n|$)',
    "Magnus Edenhill Open Source":    r'\n-\s+Initial (CBL-Mariner|Azure Linux) import from Magnus Edenhill Open Source \(license: BSD\)(\.|\n|$)',
    "NVIDIA":                         r'\n-\s+Initial (CBL-Mariner|Azure Linux) import from NVIDIA \(license: (ASL 2\.0|GPLv2)\)(\.|\n|$)',
+    "NVIDIA (BSD)":                   r'\n-\s+Initial (CBL-Mariner|Azure Linux) import from NVIDIA \(BSD\) \(license: (BSD)\)(\.|\n|$)',


No need for "CBL-Mariner"` in the new source attributions. The other ones are to support the old specs. And there's no need to repeat "BSD", I think.

Suggested change

"NVIDIA (BSD)": r'\n-\s+Initial (CBL-Mariner|Azure Linux) import from NVIDIA $BSD$ $license: (BSD)$(\.|\n|$)',

"NVIDIA (BSD)": r'\n-\s+Initial Azure Linux import from NVIDIA $license: (BSD)$(\.|\n|$)',

A bigger question, perhaps: why add a new entry specially for BSD? I see the older one for Nvidia already covers two licenses. Is there a reason we don't want to just update that one?

PawelWMS · 2025-01-15T23:57:17Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+BuildRequires:  gcc
+BuildRequires:  make
+BuildRequires:  kernel-devel = %{target_kernel_version_full}
+BuildRequires:  kernel-headers = %{target_kernel_version_full}
+BuildRequires:  binutils
+BuildRequires:  systemd
+BuildRequires:  kmod
+BuildRequires:  mlnx-ofa_kernel-devel = %{_version}
+BuildRequires:  mlnx-ofa_kernel-source = %{_version}


General comment for all signed specs: they don't need most BuildRequires, since they only expand already built RPMs and replace a few files. The packages providing rpm2cpio and cpio are part of the default build chroot but to be on the safe side, I'd add them to BuildRequires.

PawelWMS · 2025-01-15T23:57:58Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+Requires:       kmod
+
+%description
+fwctl signed kernel modules


This is the description, that the published package will have, so I'd suggest something, that gives users more information about the package.

PawelWMS · 2025-01-15T23:58:55Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+# This package's "version" and "release" must reflect the unsigned version that
+# was signed.
+# An important consequence is that when making a change to this package, the
+# unsigned version/release must be increased to keep the two versions consistent.
+# Ideally though, this spec will not change much or at all, so the version will
+# just track the unsigned package's version/release.


No need for the comment - the specs entanglement PR check will handle that for you. Just make sure to update the set of spec pairs, which need to be kept in sync.

PawelWMS · 2025-01-15T23:59:26Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+Source1:        fwctl.ko
+Source2:        mlx5_fwctl.ko
+
+BuildRoot:	/var/tmp/%{name}-%{version}-build


Is there a need to define BuildRoot explicitly here instead of using the default?

PawelWMS · 2025-01-16T00:01:10Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

+%postun
+/sbin/depmod %{KVERSION}
+
+%if "%{KMP}" != "1"


I'm not sure KMP makes much sense in the signed spec, since its only purpose is to repackage the unsigned RPM. Also, with the way it's written right now, if KMP ever was equal to 1, we'd end up with an invalid spec file (no %files section).

PawelWMS · 2025-01-16T00:02:09Z

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec

@@ -0,0 +1,142 @@
+#


I've added a bunch of comments into this spec but they apply to most/all other signed spec files.

PawelWMS · 2025-01-16T00:03:06Z

SPECS/fwctl/fwctl.spec

+License:	 GPLv2
+Url:		 http://nvidia.com
+Group:		 System Environment/Base
+Source0:         https://linux.mellanox.com/public/repo/mlnx_ofed/24.10-0.7.0.0/SRPMS/fwctl-24.10.tgz#/%{_name}-%{_version}.tgz


Please also use macros to parameterize the URL, so that it updates automatically with the macros. This makes it easier to miss necessary changes during updates.

PawelWMS · 2025-01-16T00:07:09Z

SPECS/knem/knem.spec

+%if "%{KMP}" != "1"
+%files -n %{non_kmp_pname}
+/lib/modules/%{KVERSION}/%{install_mod_dir}/
+%if %{IS_RHEL_VENDOR}
+%if ! 0%{?fedora}
+%config(noreplace) %{_sysconfdir}/depmod.d/%{_name}.conf
+%endif
+%endif
+%endif


Missing %license. Not strictly necessary when there's a signed spec counterpart to this one but consistency helps. Having said that, a file explaining the license IS necessary.

Binu Jose Philip added 2 commits December 15, 2024 20:31

Add signed specs for kernel modules in mofed and dependencies

8389830

microsoft-github-policy-service bot added the Packaging label Dec 17, 2024

Binu Jose Philip added 8 commits December 17, 2024 19:01

update cgmanifest with new packages

27600fb

fixing fwctl spec

dfe1400

fixing fwctl spec defines

28a9e64

make specs aware of host environment for kernel-headers

676535f

Add license and source verified clause. Fix tags.

04ef8ab

Update license map.

7e0bca1

License map attribution fixes

01f1c67

invert build host detection condition

0a6d155

microsoft-github-policy-service bot added Tools 3.0-dev PRs Destined for AzureLinux 3.0 labels Dec 30, 2024

Binu Jose Philip added 7 commits January 1, 2025 07:01

fix release field for kernel modules

9f8adbe

add srp spec and srp-signed spec

3d3e4df

cgmanifest changes for srp

31e88d3

add correct checksum for srp tgz

cf23ff6

license configuration for srp package

88b578d

use version specific url for srp source tgz

95fac18

revert mlx-bootctl spec change

c4a13d5

binujp marked this pull request as ready for review January 13, 2025 18:50

binujp requested a review from a team as a code owner January 13, 2025 18:50

gjswalling reviewed Jan 13, 2025

View reviewed changes

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec Outdated Show resolved Hide resolved

PawelWMS reviewed Jan 13, 2025

View reviewed changes

SPECS/ibsim/ibsim.spec Outdated Show resolved Hide resolved

PawelWMS reviewed Jan 13, 2025

View reviewed changes

gjswalling reviewed Jan 13, 2025

View reviewed changes

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec Show resolved Hide resolved

gjswalling reviewed Jan 13, 2025

View reviewed changes

SPECS/fwctl/fwctl.spec Show resolved Hide resolved

binujp and others added 3 commits January 14, 2025 11:38

Merge branch '3.0-dev' into bphilip/add-mofed-and-dependencies

109c830

Update license map.

1dffb69

License map attribution fixes

d116642

Binu Jose Philip and others added 2 commits January 14, 2025 21:13

add missing comma in licences.json

26e2829

Build MOFED and dependencies for x86_64 and fix code review comments

3ce9426

gjswalling approved these changes Jan 15, 2025

View reviewed changes

chalamalasetty added 2 commits January 14, 2025 23:29

Keep one initial version changelog entry and add license files

0cdaff3

Add license file section for SPECS

b42ac94

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec Show resolved Hide resolved

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/fwctl-signed/fwctl-signed.spec Show resolved Hide resolved

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/iser-signed/iser-signed.spec Show resolved Hide resolved

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/iser-signed/iser-signed.spec Show resolved Hide resolved

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/isert-signed/isert-signed.spec Show resolved Hide resolved

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/isert-signed/isert-signed.spec Show resolved Hide resolved

jslobodzian reviewed Jan 15, 2025

View reviewed changes

SPECS-SIGNED/knem-signed/knem-modules-signed.spec Show resolved Hide resolved

jslobodzian approved these changes Jan 15, 2025

View reviewed changes

binujp requested a review from romoh January 15, 2025 20:07

PawelWMS reviewed Jan 15, 2025

View reviewed changes

PawelWMS reviewed Jan 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bphilip/add mofed and dependencies #11479

Bphilip/add mofed and dependencies #11479

binujp commented Dec 17, 2024 •

edited

Loading

PawelWMS Jan 13, 2025

binujp Jan 14, 2025 •

edited

Loading

chalamalasetty Jan 15, 2025

PawelWMS Jan 15, 2025

chalamalasetty Jan 15, 2025

PawelWMS Jan 15, 2025

chalamalasetty Jan 15, 2025

PawelWMS Jan 15, 2025

PawelWMS Jan 15, 2025

PawelWMS Jan 15, 2025

PawelWMS Jan 15, 2025

PawelWMS Jan 15, 2025

PawelWMS Jan 16, 2025

PawelWMS Jan 16, 2025

PawelWMS Jan 16, 2025 •

edited

Loading

PawelWMS Jan 16, 2025 •

edited

Loading

	"NVIDIA (BSD)": r'\n-\s+Initial (CBL-Mariner\|Azure Linux) import from NVIDIA \(BSD\) \(license: (BSD)\)(\.\|\n\|$)',
	"NVIDIA (BSD)": r'\n-\s+Initial Azure Linux import from NVIDIA \(license: (BSD)\)(\.\|\n\|$)',

Bphilip/add mofed and dependencies #11479

Are you sure you want to change the base?

Bphilip/add mofed and dependencies #11479

Conversation

binujp commented Dec 17, 2024 • edited Loading

Merge Checklist

Summary

Change Log

Does this affect the toolchain?

Associated issues

Links to CVEs

Test Methodology

Choose a reason for hiding this comment

binujp Jan 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PawelWMS Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

PawelWMS Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

binujp commented Dec 17, 2024 •

edited

Loading

binujp Jan 14, 2025 •

edited

Loading

PawelWMS Jan 16, 2025 •

edited

Loading

PawelWMS Jan 16, 2025 •

edited

Loading