Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
bob.learn.tensorflow
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
GitLab community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
This is an archived project. Repository and other project resources are read-only.
Show more breadcrumbs
bob
bob.learn.tensorflow
Merge requests
!21
Resolve "Adopt to the Estimators API"
Code
Review changes
Check out branch
Download
Patches
Plain diff
Expand sidebar
Merged
Resolve "Adopt to the Estimators API"
40-adopt-to-the-estimators-api
into
master
Overview
21
Commits
23
Pipelines
11
Changes
3
Merged
Tiago de Freitas Pereira
requested to merge
40-adopt-to-the-estimators-api
into
master
7 years ago
Overview
17
Commits
23
Pipelines
11
Changes
3
Closes
#40 (closed)
Edited
7 years ago
by
Tiago de Freitas Pereira
0
0
Merge request reports
Viewing commit
65cdd62c
Prev
Next
Show latest version
3 files
+
296
−
1
Inline
Compare changes
Side-by-side
Inline
Show whitespace changes
Show one file at a time
Files
3
65cdd62c
Created mechanism to batch siamease data
· 65cdd62c
Tiago de Freitas Pereira
authored
7 years ago
bob/learn/tensorflow/dataset/__init__.py
+
77
−
1
View file @ 65cdd62c
Edit in single-file editor
Open in Web IDE
Show full file
import
tensorflow
as
tf
import
numpy
DEFAULT_FEATURE
=
{
'
train/data
'
:
tf
.
FixedLenFeature
([],
tf
.
string
),
'
train/label
'
:
tf
.
FixedLenFeature
([],
tf
.
int64
)}
@@ -68,4 +68,80 @@ def append_image_augmentation(image, gray_scale=False,
image
=
tf
.
image
.
per_image_standardization
(
image
)
return
image
def
siamease_pairs_generator
(
input_data
,
input_labels
):
"""
Giving a list of samples and a list of labels, it dumps a series of
pairs for siamese nets.
**Parameters**
input_data: List of whatever representing the data samples
input_labels: List of the labels (needs to be in EXACT same order as input_data)
"""
# Lists that will be returned
left_data
=
[]
right_data
=
[]
labels
=
[]
def
append
(
left
,
right
,
label
):
"""
Just appending one element in each list
"""
left_data
.
append
(
left
)
right_data
.
append
(
right
)
labels
.
append
(
label
)
possible_labels
=
list
(
set
(
input_labels
))
input_data
=
numpy
.
array
(
input_data
)
input_labels
=
numpy
.
array
(
input_labels
)
total_samples
=
input_data
.
shape
[
0
]
# Filtering the samples by label and shuffling all the indexes
indexes_per_labels
=
dict
()
for
l
in
possible_labels
:
indexes_per_labels
[
l
]
=
numpy
.
where
(
input_labels
==
l
)[
0
]
numpy
.
random
.
shuffle
(
indexes_per_labels
[
l
])
left_possible_indexes
=
numpy
.
random
.
choice
(
possible_labels
,
total_samples
,
replace
=
True
)
right_possible_indexes
=
numpy
.
random
.
choice
(
possible_labels
,
total_samples
,
replace
=
True
)
genuine
=
True
for
i
in
range
(
total_samples
):
if
genuine
:
# Selecting the class
class_index
=
left_possible_indexes
[
i
]
# Now selecting the samples for the pair
left
=
input_data
[
indexes_per_labels
[
class_index
][
numpy
.
random
.
randint
(
len
(
indexes_per_labels
[
class_index
]))]]
right
=
input_data
[
indexes_per_labels
[
class_index
][
numpy
.
random
.
randint
(
len
(
indexes_per_labels
[
class_index
]))]]
append
(
left
,
right
,
0
)
#yield left, right, 0
else
:
# Selecting the 2 classes
class_index
=
list
()
class_index
.
append
(
left_possible_indexes
[
i
])
# Finding the right pair
j
=
i
# TODO: Lame solution. Fix this
while
j
<
total_samples
:
# Here is an unidiretinal search for the negative pair
if
left_possible_indexes
[
i
]
!=
right_possible_indexes
[
j
]:
class_index
.
append
(
right_possible_indexes
[
j
])
break
j
+=
1
if
j
<
total_samples
:
# Now selecting the samples for the pair
left
=
input_data
[
indexes_per_labels
[
class_index
[
0
]][
numpy
.
random
.
randint
(
len
(
indexes_per_labels
[
class_index
[
0
]]))]]
right
=
input_data
[
indexes_per_labels
[
class_index
[
1
]][
numpy
.
random
.
randint
(
len
(
indexes_per_labels
[
class_index
[
1
]]))]]
append
(
left
,
right
,
1
)
genuine
=
not
genuine
return
left_data
,
right_data
,
labels
Loading