You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Feb 24, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Feb 24, 2026. It is now read-only.
The W2A8 fails the assert_close test. Bitblas version is 0.1.0.post
Code:
importtorchimportbitblasimporttorch.nnasnn# enabling debug outputN=4096K=2560importpicklewithopen('log.bin', 'rb') asf:
loaded_tensors=pickle.load(f)
input_tensor, weight_tensor=loaded_tensorsprint(input_tensor.size())
print(input_tensor.max())
print(input_tensor.min())
print(weight_tensor.size())
print(weight_tensor.max())
print(weight_tensor.min())
bitblas.set_log_level("Info")
matmul_config=bitblas.MatmulConfig(
# M=1, # M dimensionN=N, # N dimensionK=K, # K dimensionA_dtype="int8", # activation A dtypeW_dtype="int2", # weight W dtypeaccum_dtype="int32", # accumulation dtypeout_dtype="float32", # output dtypelayout="nt", # matrix layout, "nt" indicates the layout of A is non-transpose and the layout of W is transposewith_bias=False, # bias# configs for weight only quantizationgroup_size=None, # setting for grouped quantizationwith_scaling=False, # setting for scaling factorwith_zeros=False, # setting for zeroszeros_mode=None, # setting for how to calculating zeros
)
matmul=bitblas.Matmul(config=matmul_config)
# Transform weight tensor to int4 data typeweight_tensor_int8=matmul.transform_weight(weight_tensor)
# Perform mixed-precision matrix multiplicationoutput_tensor=matmul(input_tensor, weight_tensor_int8)
# Reference result using PyTorch matmul for comparison# ref_result = torch.matmul(input_tensor.to(torch.float32), weight_tensor.t().to(torch.float32))ref_result=nn.functional.linear(input_tensor.to(torch.float32), weight_tensor.to(torch.float32))
print("Ref output:", ref_result)
print("BitBLAS output:", output_tensor)
torch.testing.assert_close(output_tensor, ref_result, rtol=1e-2, atol=1e-0)
Oddly, when I run another go, the results match! My guess is different kernel is selected. See attached (see test.zip above) for the complete log and the data I use.
Activity
tanvoontao commentedon Jan 22, 2026
Haisong Ding (@HaisongDing) Have u figure out the issue?
HaisongDing commentedon Jan 23, 2026
No, I switched to llama.cpp for low-bit LMM inference.