angle_emb.utils

Attributes

logger

Functions

cosine_similarity(vec1, vec2)	Calculate cosine similarity between two vectors.
set_device(→ str)	Set device automatically
find_all_linear_names(→ List[str])	Find all linear layer names
get_pooling(→ torch.Tensor)	Pooling the model outputs.

Module Contents

angle_emb.utils.logger

angle_emb.utils.cosine_similarity(vec1: List[int], vec2: List[int])

Calculate cosine similarity between two vectors.

Parameters:

• vec1 – a list of integers

• vec2 – a list of integers

Returns:

a float value between 0 and 1, indicating the similarity between the two vectors.

angle_emb.utils.set_device() → str

Set device automatically

Returns:

str, device name

angle_emb.utils.find_all_linear_names(model: transformers.PreTrainedModel, linear_type: object | None = None) → List[str]

Find all linear layer names

Parameters:

• model – PreTrainedModel

• linear_type – Optional[object] = None, linear type, such as nn.Linear and bnb.nn.Linear4bit.

Returns:

List[str], linear layer names

angle_emb.utils.get_pooling(outputs: torch.Tensor, inputs: Dict, pooling_strategy: str, padding_side: str) → torch.Tensor

Pooling the model outputs.

Parameters:

• outputs – torch.Tensor. Model outputs (without pooling)

• inputs – Dict. Model inputs

• pooling_strategy – str. Pooling strategy [cls, cls_avg, cls_max, last, avg, mean, max, all, int]

• padding_side – str. Padding strategy of tokenizers (left or right). It can be obtained by tokenizer.padding_side.