Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
bob
bob.pipelines
Commits
755e6cdf
Commit
755e6cdf
authored
Aug 12, 2021
by
Amir MOHAMMADI
Browse files
[CheckpointWrapper] Use correct extension during atomic writing
parent
647cd8ad
Pipeline
#53133
passed with stage
in 16 minutes and 16 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
bob/pipelines/wrappers.py
View file @
755e6cdf
...
...
@@ -4,6 +4,7 @@ import os
import
tempfile
from
functools
import
partial
from
pathlib
import
Path
import
cloudpickle
import
dask.bag
...
...
@@ -394,11 +395,15 @@ class CheckpointWrapper(BaseWrapper, TransformerMixin):
to_save
=
getattr
(
sample
,
self
.
sample_attribute
)
for
_
in
range
(
self
.
attempts
):
try
:
dirname
=
os
.
path
.
dirname
(
path
)
os
.
makedirs
(
dirname
,
exist_ok
=
True
)
# Atomic writing
with
tempfile
.
NamedTemporaryFile
(
dir
=
dirname
,
delete
=
False
)
as
f
:
extension
=
""
.
join
(
Path
(
path
).
suffixes
)
with
tempfile
.
NamedTemporaryFile
(
dir
=
dirname
,
delete
=
False
,
suffix
=
extension
)
as
f
:
self
.
save_func
(
to_save
,
f
.
name
)
os
.
replace
(
f
.
name
,
path
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment