Release Notes

30.1.0

  • The V80, V80LL and Artena bitstreams support SR-IOV, so one card can be shared by several threads, processes or virtual machines by splitting it into up to 16 virtual functions (VFs) (see the Configuring the Accelerator for Multiple Processes section of the docs)
  • vollo-tool programs a V80 or V80LL over PCIe without AMD's ami kernel driver, which is no longer shipped with the SDK
  • vollo-torch
    • Support for non-streaming torch.nn.Conv2d and torch.nn.functional.conv2d, with per-axis stride, dilation and padding
    • Support for non-streaming torch.nn.Conv3d and torch.nn.functional.conv3d, with per-axis stride, dilation and padding
    • Support for padding on non-streaming torch.nn.Conv1d and torch.nn.functional.conv1d, which previously had to be 0
  • vollo-compiler
    • Programs record which models carry state from one inference to the next. Check it with Program.model_is_stateful.
    • Program.metrics() reports pafs_used, the number of Programmable Activation Functions each core uses, and num_paf, the number available
    • Fix clamps being dropped from slices that the compiler fuses with a concatenation
    • NumberFormat values no longer compare equal to integers (NumberFormat.BF16 == 0 used to be True) and can now be used as dictionary keys and in sets
    • VM.run methods give correct results for Fortran-ordered and other non-C-contiguous input arrays, which were previously read in memory order
    • Improve VM cycle count accuracy
    • Remove the Silicom fb4CGg3 (UltraScale+) hardware config: vollo_compiler.Config.silicom_fb4cgg3_c3b32() and its alias fb4cgg3_c3b32() are gone, and "Ultrascale" is no longer accepted as a Config fabric
  • vollo-rt
    • vollo_rt_add_job and the other vollo_rt_add_job_* functions may start a job straight away, instead of only on the next vollo_rt_poll. Set VOLLO_SUBMIT_ON_ADD=0 to start jobs only on vollo_rt_poll, as before.
    • Fix libvollo_rt.a and libvollo_cfg.a static libraries to only export their C API
  • Bitstreams
    • The V80 and V80LL bitstreams link at PCIe Gen5 x8 (32 GT/s) when in a slot that supports it
    • Reduce the V80, V80LL and Artena bitstreams' host PCIe address space requirements
    • The V80, V80LL and Artena bitstreams protect the card from overheating by holding new jobs above 100 °C (95 °C on the Artena) until it has cooled
  • IP Core
    • The Vollo IP Core ships vollo_ip_core.sdc to constrain its internal clock-domain crossings (see the Quartus Integration section of the IP Core docs)

29.1.0

  • Support for torch.expm1 and torch.special.expm1 with special handling for inputs near zero
  • Improve accuracy of tanh for inputs near zero
  • Increase number of Programmable Activation Functions on Versal boards from 4 to 8
  • Speed up model compilation

29.0.0

  • Support for torch.log, torch.log2, torch.log10 (as well as respective torch.Tensor methods) and the ONNX Log operator, at BF16 precision
  • Bump the amd_v80_c6b32, amd_v80ll_c6b32, silicom_artena_c8b32 and silicom_artena_c8b32lstm cards' cell_state_depth to 2048
  • Recurrent models such as LSTMs use fewer tensor descriptors. This allows a larger number of these models to be combined into a multi-model program.
  • The example applications default --device to the VOLLO_CARD_BDF environment variable when it is set, so a host with several accelerators can select one without editing every command
  • vollo-tool fpga-config can name a card by PCI BDF as well as by index
  • Report loading a .vollo program from a mismatched compiler version as a vollo_compiler.LoadError explaining the version difference, instead of a panic
  • Raise ValueError instead of panicking when constructing a Config with an unknown fabric or a missing clock_mhz
  • Program.load_bytes now raises vollo_compiler.LoadError rather than ValueError when the data isn't a valid program, matching Program.load
  • Say what produced a program file that wasn't written by the Vollo compiler, when it records that, instead of reporting it as an empty field
  • Fix a bug which caused programs built through the ProgramBuilder to produce incorrect values if they used dynamic weights
  • Fix an operator fusion bug in where operations which could select the wrong branch depending on the operations on the inputs to the where operation
  • Fixed panic when a dynamic weight matmul is an input to a sum
  • Fix miscompilation with optimize_transforms = False for summing values from different timesteps
  • Fix clamp behaviour not matching PyTorch / ONNX when max < min
  • Fix streaming transform panics
  • Fix handling of BatchNorm1d bias
  • Fix panics involving sums
  • Fix compiler causing stack overflow
  • Fix rare LSTM miscompilation

28.1.2

  • Fix a bug which caused programs built through the ProgramBuilder with certain activation functions to produce incorrect values

28.1.1

  • Fix vollo_torch.nn.LSTMCell using a different cell state each time it's called
  • Fix a bug which caused ProgramBuilder.to_program to fail on programs with fp8 weights
  • Reduce tensor descriptors used in multi-model programs, allowing more models to be combined into a program
  • Fix a multi-model program intermittently running the wrong model, from some point in a run until the program is reloaded. This affected inputs transferred over MMIO (the default for small inputs) and only on some host CPUs
  • Add an example in vollo-ip-sim for using your own data path
  • Remove the automatic FPGA temperature monitoring added in 28.1.0. It sometimes caused a crash with the artena bitstream. VOLLO_RT_TEMP_LIMIT_C and VOLLO_RT_TEMP_WARN_C no longer do anything. The monitor will be reinstated in the next hardware release with the necessary bitstream change.

28.1.0

  • Add automatic FPGA temperature monitoring in Vollo RT: it warns when the device gets hot and stops compute before it overheats, configurable with the VOLLO_RT_TEMP_LIMIT_C and VOLLO_RT_TEMP_WARN_C environment variables
  • Remove the unused DDR4 DIMM memory controller from the V80 and V80LL bitstreams
  • Speed up model compilation
  • Fix a DMA deadlock that could occur when a model output transfer was paused partway through
  • Fix the input size threshold used to decide whether inputs are sent over MMIO
  • Fix a stall in models with multiple MMIO inputs when an optional input is empty
  • Prevent the SDK installer from opening an X11 terminal window when run non-interactively

28.0.3

  • Fix a multi-model program intermittently running the wrong model, from some point in a run until the program is reloaded. This affected inputs transferred over MMIO (the default for small inputs) and only on some host CPUs

28.0.2

  • Fix a DMA buffer allocation error in vollo-rt when running many concurrent jobs with large inputs
  • Fix an issue in vollo-rt when running many concurrent jobs with MMIO inputs

28.0.1

  • Add support for torch.nn.Flatten
  • Add support for torch.nn.MaxPool1d, torch.nn.functional.max_pool1d
  • Matrix multiplication optimisations

28.0.0

  • MMIO optimisations
  • Speed up model compilation
  • Optimise pointwise operations
  • Support partial updates with inputs that have more than 65536 elements
  • Support for transposed dynamic weight matmuls
  • Add functions to check if a model was compiled with generate_state_reset = True
    • vollo_rt_model_is_resettable in Vollo RT C/C++ API
    • vollo_rt.VolloRTContext.model_is_resettable in Vollo RT Python bindings
    • vollo_compiler.Program.model_is_resettable in vollo-compiler
  • Add vollo_rt_add_device in Vollo RT C/C++ API as a more versatile version of vollo_rt_add_accelerator
    • vollo_rt.VolloRTContext.add_device in Vollo RT Python bindings
  • Handle SIGINT in long-running vollo_compiler functions
  • Support for non-streaming torch.nn.Conv1d
  • Support torch.sum/torch.Tensor.sum over multiple dimensions
  • Support variable-length arg for dims in torch.Tensor.permute
  • Fix handling of exponentiation on infinities

27.1.0

  • State reset with vollo_rt_add_reset_job (for models compiled with generate_state_reset = True)
  • Support for non-zero LSTM initial state
  • Support GELU (exact and tanh approximation)
  • Faster SiLU
  • Improve operator fusion in some cases
  • Fix a bug handling strides in slices
  • Fix HwConfig info (vollo-tool read-hw-config) of Artena bitstreams
  • Standardize naming of vollo_compiler.Config presets

27.0.1

  • Reduce tensor ram usage in certain models
  • Add experimental vollo_rt_prepare_raw_buffer_output_completion and vollo_rt_check_raw_buffer_output_completion functions to Vollo RT for completion detection from a different thread to the one holding the vollo_rt_context

27.0.0

  • IO optimization when using MMIO
  • Add experimental support for specifying which cores to allocate PyTorch operations to using vollo_torch.CorePartition
  • Add support for specifying which cores to allocate models to in a multi-model program by passing core_indices to vollo_compiler.ProgramBuilder.add_nnir
  • Optimize sigmoid and SiLU activation functions
  • Improve spaced latency for some stateful models that use dynamic weights
  • Reduce tensor RAM usage of state in stateful models
  • vollo_torch.Fp8Weights now errors if used on operations which require bf16 weights, such as dynamic weights

26.2.0

  • Support for FP8 (E4M3) weights on Versal devices using vollo_torch.Fp8Weights context manager
  • Support for torch.exp, torch.exp2 at FP32 precision using vollo_torch.Fp32Activations context manager
  • Support for matmul operations where both inputs are dynamic (non-constant) tensors, when using allow_dynamic_weights
  • Optimize accumulations on Versal devices, improving performance of layers such as LayerNorm and RMSNorm, and Linear layers with small output features
  • Add support for multiple state tensors in vollo_torch.nn.Scan
  • Add allow_unserializable flag to vollo_compiler.NNIR.to_program for testing programs which can't be serialized
  • Fix multi-model programs that use dynamic weights

26.1.2

  • Optimize handling of biases in Linear layers when using allow_dynamic_weights
  • Speed up model compilation
  • Add random_seeds argument to vollo_compiler.NNIR.to_program

26.1.1

  • Fix V80LL initialization bitstream so that the V80LL memory can be flashed over JTAG
  • Optimize handling of biases in Linear layers when using allow_dynamic_weights
  • Add support for multiple outputs to vollo_torch.nn.Scan
  • Speed up loading .vollo programs

26.1.0

  • Fix DMA bug introduced on V80 in Vollo SDK 26.0.0
  • Add Alveo V80LL bitstream and vollo_compiler.Config.v80ll_c6b32 hardware config
  • Add support for Linear layers where the contracted dimension is not the data dimension via the allow_dynamic_weights flag for vollo_compiler.NNIR.to_program
  • Add support for multiple inputs to vollo_torch.nn.Scan
  • Add support for indexing with negative indices in: torch.stack, torch.sum, torch.permute, torch.squeeze, torch.unsqueeze
  • Add support for torch.nn.functional.linear
  • Add optional bias argument to vollo_torch.nn.PaddedConv1d

26.0.2

  • Update example/partial_update.c to allow multiple inputs and mixed precision
  • Fix bug in FP32/multi-input partial updates
  • Speed up model compilation

26.0.1

  • Make vollo-tool license use the system's CA certificates
  • Fix bug in FP32 partial updates

26.0.0

  • V80 DMA optimizations
  • Support for a subset of operations at FP32
    • vollo-torch
      • Add vollo_torch.Fp32Activations context manager
      • Add inputs_precisions and output_precisions arguments to vollo_torch.fx.nnir.to_nnir
    • vollo-compiler:
      • Add model_input_number_format and model_output_number_format methods to vollo_compiler.Program
      • Add vollo_compiler.NumberFormat enum
    • vollo-rt C/C++ API
      • Add vollo_rt_add_job, vollo_rt_add_job_partial_update, vollo_rt_model_input_format, vollo_rt_model_output_format, vollo_rt_get_raw_buffer_bytes functions and number_format enum
    • vollo-rt Python bindings
      • Add add_job, add_job_f32, model_output_format methods to vollo_rt.VolloRTContext
  • Memory usage and compilation time improvements in the compiler
  • Add quick_compile flag to vollo_compiler.NNIR.to_program for faster compilation
  • Add max_sparse_entries option to vollo_compiler.NNIR.to_program to configure the number of nonzero entries allowed in weights for non-standard memory format MatMuls
  • Add token-info subcommand to vollo-tool license to show information about a purchase token
  • Add info message to vollo-tool license redeem-device if the device being redeemed for has been redeemed on an expired or nearly expiring token

25.1.2

  • Reduced memory usage of the compiler during compilation of large models
  • Improve ami-tool's detection of bitstream UUIDs
  • Improve AMI driver's compatibility with Linux kernel versions

25.1.1

  • Fix bug in vollo-tool where vollo-tool fpga-config did not enumerate the V80 management physical function
  • Fix bug in load-kernel-driver.sh where the Vollo driver was loaded for the V80 management physical function instead of the AMI driver

25.1.0

  • Early access support for Mamba models
  • Add support for SiLU
  • Add support for Softplus
  • Add support for Exp, Exp2
  • Add support for Sigmoid
  • Add support for Softmax
  • Speed up model compilation

25.0.1

  • Add support for size 0 input and output tensors in the compiler
  • Add vollo_rt_model_input_shape_len and vollo_rt_model_output_shape_len functions to Vollo RT
  • Improve constant folding in the compiler

25.0.0

  • Add early access support for Silicom fb4CGg3@VU09P
  • "Spaced" inference latency optimization now applies to multi-model programs too
  • Increase clock speed on V80 from 280MHz to 300MHz.
  • Compiler and scheduler optimizations to improve inference latency
  • Speed up model compilation
  • Add support for bidirectional LSTMs

24.0.1

  • Improve latency of streaming models, especially e.g. large LSTMs, when inferences have spacing between them, i.e. not being run back-to-back
  • Add spaced flag to vollo_compiler.Program.cycle_count_per_inference to get cycle counts of programs with or without spacing in between inferences
  • Speed up model compilation

24.0.0

  • Improved latency when using multiple models
  • Improved V80 performance
  • Scheduler improvements, especially for models with large layers
  • remember_allocations argument added to vollo_compiler.ProgramBuilder to help fit more models in multi-model programs

23.1.0

  • Add initial support for Alveo V80, further performance optimisations still outstanding
  • Add support for Napatech NT400D11
  • Add support for vfio-pci; use load-kernel-driver.sh vfio to load it, required for V80
  • Add lock to Vollo RT to prevent concurrent usage of the accelerator
  • Improve VM cycle count estimates for Agilex devices
  • Additional model support:
    • Add support for broadcasting non-constant tensors except along the data dimension
    • Add grouped convolution support to vollo_torch.nn.PaddedConv1d
    • Add support for reshape operations
    • Changes to the API of vollo_torch.nn.Scan: the step function now returns an output tensor and a separate state tensor instead of a single tensor; the forward method now takes both an input_axis and an output_axis instead of a single axis argument

22.1.0

  • Update compatibility with newer IA420F boards (IA420F-0015)

22.0.1

  • Allow weights to be shared in multi-model programs

22.0.0

  • Add support for compiling models with multiple input tensors and multiple output tensors
  • Improve accuracy of LSTM unit
  • Change behavior of VOLLO_FP32_ROUND in Vollo RT so that it's enabled by default; set to 0 to truncate f32 inputs
  • Change vollo-tool reset --pci to vollo-tool reset-pci
  • Expand supported PyTorch stacking and concatenating operations: concatenate, stack, vstack, hstack, row_stack, column_stack, dstack
  • Expand supported PyTorch transposition operations: permute, swapdims, swapaxes, t, T, mT

21.1.1

  • Initial support for torch.nn.LSTM
  • Performance improvements in VM simulation, especially for LSTMs
  • Improve error messages from vollo_torch.fx.nnir.to_nnir for unsupported field accesses (getattr) in PyTorch model
  • Add f32_round argument to vollo_compiler.VM.run methods to choose whether to round or truncate f32 inputs (previously always rounded)
  • Fix handling of non-contiguous input arrays/tensors in Vollo RT Python bindings
  • Fix bug in streaming_transform for tensor sum reductions

21.1.0

  • Support tanh

21.0.0

  • Runtime/bitstream optimisation for small inputs (using MMIO instead of DMA)
  • Scheduling and architecture optimisations
  • Add reset subcommand to vollo-tool
  • Support ReLU via torch.relu

20.0.3

  • Separate bitstreams from Vollo SDK
  • Add c2b64d hw config to support models up to 8M parameters (bitstream and compiler)
  • Improve compiler error messages
  • Fix example build

20.0.2

  • Fix for incorrect vollo_rt_accelerator_num_cores introduced in 20.0.1

20.0.1

  • vollo_rt_add_vm to test the vollo-rt API without an accelerator
  • vollo_rt_load_program_from_buffer and vollo_compiler.Program.{save,load}_bytes
  • Add vollo_torch.nn.RecurrentStateLSTM for modelling streaming LSTM models across forward passes
  • Codegen fix for vollo_torch.nn.Scan
  • Fix incorrect input validation for torch.sum layers
  • Change vollo-rt example to compile with older C compilers

20.0.0

  • Add support for LayerNorm
  • Add support for RMSNorm
  • Add support for sqrt operations (torch.sqrt and torch.rsqrt)
  • Add support for summing over the data dimension
  • Add cycle_count_per_inference and compute_duration_per_inference_us Program methods
  • Add support for a wider range of torch arithmetic operation aliases

19.2.3

  • Downgrade glibc dependency to support systems with glibc >=2.17

19.2.2

  • Add support for torch.div, torch.Tensor.div
  • Fix compiler code generation bug for division

19.2.1

  • Add support for scalars on the left of division
  • Add support for Reciprocal node in ONNX frontend

19.2.0

  • Add support for division by non-constant tensors
  • Fix slicing in ONNX frontend

19.1.1

  • Fix compiler bug in constant folding

19.1.0

  • Add support for partial updates of input data on the accelerator
  • VM simulates Vollo accelerator bit-accurately: bf16_precision argument renamed to bit_accurate and enabled by default
  • vollo-tool includes license self-service
  • Performance improvements due to DMA optimization

18.0.2

  • Add optimize_transforms option to the compiler to improve program schedule in some cases

18.0.1

  • Add fallback to Vollo RT and vollo-tool for when AVX is not available

18.0.0

  • Vollo RT support for using raw DMA buffers to skip IO copy
  • Vollo RT remove redundant/noisy warnings on error: it is the user's responsibility to check returned errors
  • Compiler optimization for Where nodes
  • Compiler scheduling optimizations
  • Vollo IP Core public documentation

0.17.1

  • Fix vollo-tool compatibility with older bitstreams

0.17.0

  • New DMA engine that reduces IO latencies by ~1.3us
  • Initial support for non-streaming LSTM

0.16.0

  • Vollo IP Core now available on request
  • Add C library for configuring IP Core: vollo-cfg
  • Support for slicing/concatenation in the middle of models
  • Support for BatchNorm nodes
  • Support for Scan/LSTMCell nodes
  • Add --io-only option to vollo-onnx
  • Add program-metadata command to vollo-tool
  • Fix compiler bug with transposing streaming dimension
  • Fix accelerator bug in initial state of streaming models

0.15.0

  • Accelerator bug fix

0.14.0

  • Support for filtering dropout layers
  • Instruction packing improvements
  • LSTM performance improvement
  • Improvements to weight sharing

0.13.0

  • Support for multi-model programs
  • Provide Python bindings to Vollo RT: vollo_rt
  • Improved support and error messages for tensor indexing in compiler
  • The unweave transform is now automatic

0.12.2

  • Support for LSTM nodes in ONNX frontend
  • Support for squeezing, unsqueezing, reduce sum, using unweave transformation
  • Improved error reporting in vollo_torch lowering to NNIR

0.12.1

  • vollo-torch fix type hints being incompatible with Python 3.7/3.8
  • vollo-rt.h fix namespacing issue (error_t -> vollo_rt_error_t)
  • Runtime optimisations
  • Added IO only benchmarks

0.12.0

  • Initial support for ONNX models in compiler
  • Support for LSTM nodes
  • Improved error reporting in compiler
  • Compiler API changes
  • New runtime API with access to model metadata
  • HW optimisations (pointwise operations)
  • IA840F support

0.10.1

  • Support for scalar (int, float) literals in pointwise operations in vollo-torch.

0.10.0

  • Architectural changes in bitstream to support compiler
  • Reduced latency from reduced core to core communication in the bitstream
  • Add general model compiler and VM simulation with Python bindings in vollo-python
  • Add PyTorch frontend to model compiler in vollo-torch