From c98e098fd641c88c7c4493f46d6ac73f3384c48a Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Tue, 19 Dec 2023 10:28:08 +0000 Subject: [PATCH] build based on 7e934f3 --- dev/advanced/index.html | 2 +- dev/compression/index.html | 2 +- dev/customserialization/index.html | 2 +- dev/hdf5compat/index.html | 2 +- dev/index.html | 2 +- dev/internals/index.html | 2 +- dev/legacy/index.html | 2 +- dev/search/index.html | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/dev/advanced/index.html b/dev/advanced/index.html index f6a54a10..26be9076 100644 --- a/dev/advanced/index.html +++ b/dev/advanced/index.html @@ -43,4 +43,4 @@ # to an `Upgrade` instance with the new struct. load("test.jld2", "data"; typemap=Dict("Main.OldStructVersion" => JLD2.Upgrade(UpdatedStruct)))

Groups - Appending to files

Group objects can be constructed with two optional keyword arguments:

g = Group(file;
           est_num_entries=4
-          est_link_name_len=8)

These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.

JLD2DebugTools

There is an experimental repository JLD2DebugTools.jl that may help with debugging files.

Fallback Behaviour

By default JLD2 will attempt to open files using the MmapIO backend. If that fails, it retries using IOStream.

+ est_link_name_len=8)

These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.

JLD2DebugTools

There is an experimental repository JLD2DebugTools.jl that may help with debugging files.

Fallback Behaviour

By default JLD2 will attempt to open files using the MmapIO backend. If that fails, it retries using IOStream.

diff --git a/dev/compression/index.html b/dev/compression/index.html index 76297895..d62a27c2 100644 --- a/dev/compression/index.html +++ b/dev/compression/index.html @@ -11,4 +11,4 @@ # Don't compress this write(f, "large_array", rand(10000)) -end +end diff --git a/dev/customserialization/index.html b/dev/customserialization/index.html index a9179669..6b4ee2fe 100644 --- a/dev/customserialization/index.html +++ b/dev/customserialization/index.html @@ -20,4 +20,4 @@ arr = [B(rand()) for i=1:10] -@save "test.jld2" arr

In this example JLD2 converts the array of B structs to a plain Vector{Float64} prior to storing to disk.

+@save "test.jld2" arr

In this example JLD2 converts the array of B structs to a plain Vector{Float64} prior to storing to disk.

diff --git a/dev/hdf5compat/index.html b/dev/hdf5compat/index.html index 81c32e45..9d91a883 100644 --- a/dev/hdf5compat/index.html +++ b/dev/hdf5compat/index.html @@ -56,4 +56,4 @@ } } }

We can see that the file contains two things at top-level. There is a dataset "a" (that is what we wanted to store) and there is a group _types which is where all the necessary type information is stored.

You can see that JLD2 committed two compound datatypes. The first one is Core.Datatype which at first seems rather unintuitive. It is needed to tell HDF5 what a serialized julia datatype looks like (a name and a list of parameters).

Below that is the definition of MyCustomStruct with two fields H5T_STD_I64LE "x" and H5T_IEEE_F64LE "y" defining the integer field x and the float field y.

A note on pointers

In the julia programming language pointers Ptr are not needed very often. However, when binary dependencies come into play and memory is passed back and forth, pointers do become relevant. Pointers are addresses to locations in memory and thus lose their meaning after a program has terminated.

In principle, there is little point in storing a pointer to a file but in order to allow for a more seamless experience JLD2 will, similar to Base.Serialization silently accept pointers. This is useful when storing large structures such as a DifferentialEquations.jl solution object that might contain a pointer somewhere. Upon deserialization any pointer fields are instantiated as null pointers.

This is done with just three lines of code utilizing the custom serialization logic and it is shown here as it serves as a good example for usage of that feature.

   writeas(::Type{<:Ptr}) = Nothing
-   rconvert(::Type{Ptr{T}}, ::Nothing) where {T} = Ptr{T}()

Usually one would also have to define a method for wconvert. However, in this case JLD2 figures out that no explicit conversion is needed to construct nothing.

+ rconvert(::Type{Ptr{T}}, ::Nothing) where {T} = Ptr{T}()

Usually one would also have to define a method for wconvert. However, in this case JLD2 figures out that no explicit conversion is needed to construct nothing.

diff --git a/dev/index.html b/dev/index.html index 3188d604..b19e22f3 100644 --- a/dev/index.html +++ b/dev/index.html @@ -64,4 +64,4 @@ julia> f["a"] # a new copy is loaded from the file 2-element Vector{Float64}: 0.0 - 0.0

Cross-compatibility

JLD2 tries to write files in a way that allows you to load them on different operating systems and in particular both on 32bit and 64bit systems. However, many julia structs may be inherently different on different architectures making this task impossible. In particular, moving data from a 64bit system to a 32bit system is only guaranteed to work for basic datatypes.

Security

Beware of opening JLD2 files from untrusted sources. A malicious file may execute code on your computer. See e.g. here. To check a file, you can use JLD2DebugTools.jl to view what kinds of objects are stored.

+ 0.0

Cross-compatibility

JLD2 tries to write files in a way that allows you to load them on different operating systems and in particular both on 32bit and 64bit systems. However, many julia structs may be inherently different on different architectures making this task impossible. In particular, moving data from a 64bit system to a 32bit system is only guaranteed to work for basic datatypes.

Security

Beware of opening JLD2 files from untrusted sources. A malicious file may execute code on your computer. See e.g. here. To check a file, you can use JLD2DebugTools.jl to view what kinds of objects are stored.

diff --git a/dev/internals/index.html b/dev/internals/index.html index ec620f9d..6cc3d505 100644 --- a/dev/internals/index.html +++ b/dev/internals/index.html @@ -31,4 +31,4 @@ save_object("example.jld2", hello)source
JLD2.shorttypestring — Method
shorttypestring(::Type{ <:UnknownType})

Convert an UnknownType to a corresponding string. This is only used to create names for reconstructed types. See also typestring.

source
JLD2.skip_to_aligned! — Function
skip_to_aligned!(io, rel=0)

Skip to nearest position aligned to a multiple of 8 bytes relative to rel.

source
JLD2.symbol_length — Method
symbol_length(x::Symbol)

Returns the length of the string represented by x.

source
JLD2.typestring — Method
typestring(::Type{ <:UnknownType})

Convert an UnknownType to a corresponding string. This is only used for warning during reconstruction errors. See also shorttypestring.

source
JLD2.write_link — Method
write_link(cio, name, offset)

Write a link message at current position in cio.

source
JLD2.@load — Macro
@load filename var1 [var2 ...]

Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.

For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.

Example

To load the variables hello and foo from the file example.jld2, use

@load "example.jld2" hello foo
source
JLD2.@save — Macro
@save filename var1 [var2 ...]
 @save filename {compress=true} var1 name2=var2

Write one or more variables var1,... from the current scope to a JLD2 file filename.

For interactive use you can save all variables in the current module's global scope using @save filename. More permanent code should prefer the explicit form to avoid saving unwanted variables.

Example

To save the string hello and array xs to the JLD2 file example.jld2:

hello = "world"
 xs = [1,2,3]
-@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source
+@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source diff --git a/dev/legacy/index.html b/dev/legacy/index.html index 05a72c28..e8f7c650 100644 --- a/dev/legacy/index.html +++ b/dev/legacy/index.html @@ -6,4 +6,4 @@ @save "example.jld2" {compress=true} hello bar=foo
JLD2.@save — Macro
@save filename var1 [var2 ...]
 @save filename {compress=true} var1 name2=var2

Write one or more variables var1,... from the current scope to a JLD2 file filename.

For interactive use you can save all variables in the current module's global scope using @save filename. More permanent code should prefer the explicit form to avoid saving unwanted variables.

Example

To save the string hello and array xs to the JLD2 file example.jld2:

hello = "world"
 xs = [1,2,3]
-@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source
JLD2.@load — Macro
@load filename var1 [var2 ...]

Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.

For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.

Example

To load the variables hello and foo from the file example.jld2, use

@load "example.jld2" hello foo
source
+@save "example.jld2" hello xs

For passing options to the saving command use {}

@save "example.jld2" {compress=true} hello xs

For saving variables under a different name use regular assignment syntax

@save "example.jld2" greeting=hello xarray = xs
source
JLD2.@load — Macro
@load filename var1 [var2 ...]

Load one or more variables var1,... from JLD2 file filename into the current scope and return a vector of the loaded variable names.

For interactive use, the form @load "somefile.jld2" will load all variables from "somefile.jld2" into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.

Example

To load the variables hello and foo from the file example.jld2, use

@load "example.jld2" hello foo
source
diff --git a/dev/search/index.html b/dev/search/index.html index f9e8985b..40ba31db 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · Julia Data Format

Loading search...

    +Search · Julia Data Format

    Loading search...