Comparing only samples of specific classes #341
-
|
Of the examples I've checked so far, they all involve creating models that compare all classes of samples from the dataset to one another, however I have a slightly different use-case that arises as a result of my current dataset structure. In my case, I have a dataset of objects that looks something like so (with each and I'd like to compare only classes from within the same scene during training. The reason for this being that I cannot guarantee that each of the classes across scenes corresponds to the same thing, i.e. In fact, it's incredibly unlikely that they're the same thing, but technically possible, so I'd like to avoid cross- As a complete novice to metric learning, I was wondering if this type of problem has a name (as I've been unable to find anything so far), and if anyone could provide advice on where I should start looking in order to modify for example the |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 6 replies
-
|
If You could do this in your Dataset class, so you don't have to change the folders. What I mean is something like this: |
Beta Was this translation helpful? Give feedback.
-
|
Here's an approach that might work. It assumes your labels are 2-dimensional, where the first dim is the class label and the second dim is the scene label. (Warning: I haven't tested this code) from pytorch_metric_learning.utils import loss_and_miner_utils as lmu
for data, labels in dataloader:
class_labels = labels[:,0]
scene_labels = labels[:,1]
embeddings = model(data)
loss = 0
# collect all pairs per scene
for s in torch.unique(scene_labels):
mask = scene_labels == s
curr_embeddings = embeddings[mask]
curr_class_labels = class_labels[mask]
curr_pairs = lmu.get_all_pairs_indices(curr_class_labels)
loss = loss_fn(curr_embeddings, curr_class_labels, curr_pairs)
loss.backward()The only downside of this method is that you can't use a miner. However, you can use a threshold reducer in the loss function, which has a similar effect as a miner. |
Beta Was this translation helpful? Give feedback.
-
|
@KevinMusgrave Thanks so much, this is a really helpful starting point. I'm a little surprised that this problem isn't more common, but I couldn't find anyone else with a similar use-case online at all! |
Beta Was this translation helpful? Give feedback.
-
|
I guess "hierarchical metric learning" could be related |
Beta Was this translation helpful? Give feedback.
-
|
I'll look into it, thanks again! I love how well documented this repo is by the way, really amazing work! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! 😄 |
Beta Was this translation helpful? Give feedback.
If
scene_0/class_0doesn't share any characteristics ofscene_1/class_0, how about just renumbering the classes?You could do this in your Dataset class, so you don't have to change the folders.
What I mean is something like this: