update import_sstpb and fix go & rust docker image build #1224

lance6716 · 2024-02-20T06:33:04Z

this PR still has C++ make entry error

$ ./scripts/docker-run.sh make c++ 
mkdir -p /home/lance/Projects/kvproto/bin
/home/lance/Projects/kvproto/scripts/check.sh
go: modules disabled by GO111MODULE=off; see 'go help modules'
mkdir -p kvprotobuild && cd kvprotobuild && cmake ../cpp -DCMAKE_PREFIX_PATH=$GRPC_INSTALL_PATH && make
-- Using gRPC:  : /grpcinstall/v1_26_0/include, /grpcinstall/v1_26_0/lib/libgrpc.a;/grpcinstall/v1_26_0/lib/libgrpc++.a;/grpcinstall/v1_26_0/lib/libcares.a, /grpcinstall/v1_26_0/bin/grpc_cpp_plugin
CMake Error at cmake/find_abseil-cpp.cmake:1 (find_package):
  By not providing "Findabsl.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "absl", but
  CMake did not find one.

  Could not find a package configuration file provided by "absl" with any of
  the following names:

    abslConfig.cmake
    absl-config.cmake

  Add the installation prefix of "absl" to CMAKE_PREFIX_PATH or set
  "absl_DIR" to a directory containing one of the above files.  If "absl"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:26 (include)


-- Configuring incomplete, errors occurred!
See also "/home/lance/Projects/kvproto/kvprotobuild/CMakeFiles/CMakeOutput.log".
See also "/home/lance/Projects/kvproto/kvprotobuild/CMakeFiles/CMakeError.log".
make: *** [Makefile:27: c++] Error 1

I don't know how to fix it.

Signed-off-by: lance6716 <[email protected]>

YuJuncen · 2024-02-20T06:42:59Z

proto/import_sstpb.proto

@@ -283,6 +283,7 @@ message Pair {
 message WriteBatch {
    uint64 commit_ts = 1;
    repeated Pair pairs = 2;
+    uint64 start_ts = 3;


Would you add some comments about this field? I guess the start_ts is more like a hint that needed to be provided while uploading a SST in write CF.

(Also I guess such field should also be stored in SstMeta somehow. Because the observer can only get information about the SST in SstMeta.)

I can move start_ts into SstMeta, do you think it's enough? just to avoid the two places have different values.

Signed-off-by: lance6716 <[email protected]>

YuJuncen

Rest LGTM

YuJuncen · 2024-02-20T09:43:38Z

proto/import_sstpb.proto

@@ -283,6 +283,7 @@ message Pair {
 message WriteBatch {
    uint64 commit_ts = 1;
    repeated Pair pairs = 2;
+    uint64 start_ts = 3;


(Also I guess such field should also be stored in SstMeta somehow. Because the observer can only get information about the SST in SstMeta.)

overvenus · 2024-02-20T11:30:15Z

scripts/check.sh

@@ -20,8 +20,7 @@ check-protos-compatible() {
    export PATH=$GOPATH/bin:$PATH

    if [ ! -f "$GOPATH/bin/protolock" ]; then
-        GO111MODULE=off go get github.com/nilslice/protolock/cmd/protolock
-	    GO111MODULE=off go install github.com/nilslice/protolock/cmd/protolock
+	    GO111MODULE=off go install github.com/nilslice/protolock/cmd/protolock@latest


Please specify protolock version to keep proto.lock consistent.

Signed-off-by: lance6716 <[email protected]>

overvenus

To avoid making unnecessary changes to kvproto, let's hold this PR until everything is working as expected.

commit c5325a8 Author: hillium <[email protected]> Date: Tue Feb 20 18:38:01 2024 +0800 added more comments Signed-off-by: hillium <[email protected]> commit e6198f3 Author: hillium <[email protected]> Date: Mon Feb 19 16:17:21 2024 +0800 added file format for pitr Signed-off-by: hillium <[email protected]> Signed-off-by: hillium <[email protected]>

Signed-off-by: lance6716 <[email protected]>

overvenus · 2024-02-26T07:33:08Z

proto/brpb.proto

@@ -644,6 +644,11 @@ enum FileType {
    Put = 1;
 }

+enum FileFormat {
+    KvStream = 0;


enum tag 0 should be unknown or unspecified in proto3.

Please follow Proto Best Practices, thanks! https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum

Signed-off-by: hillium <[email protected]>

…art-ts

…t-ts

…)" This reverts commit 3fa61fc.

) * Revert "update import_sstpb and fix go & rust docker image build (#1224)" This reverts commit 3fa61fc. * some scripts don't need to be reverted Signed-off-by: lance6716 <[email protected]> --------- Signed-off-by: lance6716 <[email protected]>

wuhuizuo · 2024-04-01T11:25:43Z

the CI is broken:

update import_sstpb and fix go & rust docker image build

bbe58d6

Signed-off-by: lance6716 <[email protected]>

YuJuncen reviewed Feb 20, 2024

View reviewed changes

address comment

e0ee9d1

Signed-off-by: lance6716 <[email protected]>

YuJuncen reviewed Feb 20, 2024

View reviewed changes

overvenus reviewed Feb 20, 2024

View reviewed changes

lance6716 added 2 commits February 21, 2024 09:48

address comment

bf55f2e

Signed-off-by: lance6716 <[email protected]>

address comment

1421347

Signed-off-by: lance6716 <[email protected]>

overvenus reviewed Feb 21, 2024

View reviewed changes

YuJuncen and others added 3 commits February 23, 2024 16:11

Merge branch 'master' of github.com:pingcap/kvproto into start-ts

a053348

Signed-off-by: lance6716 <[email protected]>

fix git conflict

df7de98

Signed-off-by: lance6716 <[email protected]>

YuJuncen approved these changes Feb 26, 2024

View reviewed changes

overvenus reviewed Feb 26, 2024

View reviewed changes

YuJuncen and others added 4 commits February 26, 2024 17:46

address comments

7d85cf9

Signed-off-by: hillium <[email protected]>

Merge branch 'start-ts' of https://github.com/pingcap/kvproto into st…

3b2b777

…art-ts

Merge branch 'master' of https://github.com/pingcap/kvproto into star…

2b16872

…t-ts

Merge branch 'master' into start-ts

f2901cc

disksing merged commit 3fa61fc into master Mar 21, 2024
4 checks passed

disksing deleted the start-ts branch March 21, 2024 06:26

lance6716 added a commit that referenced this pull request Mar 21, 2024

Revert "update import_sstpb and fix go & rust docker image build (#1224…

03d3831

…)" This reverts commit 3fa61fc.

lance6716 mentioned this pull request Mar 21, 2024

Revert "update import_sstpb and fix go & rust docker image build" #1236

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update import_sstpb and fix go & rust docker image build #1224

update import_sstpb and fix go & rust docker image build #1224

lance6716 commented Feb 20, 2024 •

edited

Loading

YuJuncen Feb 20, 2024

lance6716 Feb 20, 2024

YuJuncen Feb 20, 2024

lance6716 Feb 20, 2024

lance6716 Feb 21, 2024

YuJuncen left a comment

YuJuncen Feb 20, 2024

overvenus Feb 20, 2024

lance6716 Feb 21, 2024

overvenus left a comment

overvenus Feb 26, 2024 •

edited

Loading

wuhuizuo commented Apr 1, 2024

update import_sstpb and fix go & rust docker image build #1224

update import_sstpb and fix go & rust docker image build #1224

Conversation

lance6716 commented Feb 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

YuJuncen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

overvenus left a comment

Choose a reason for hiding this comment

overvenus Feb 26, 2024 • edited Loading

Choose a reason for hiding this comment

wuhuizuo commented Apr 1, 2024

lance6716 commented Feb 20, 2024 •

edited

Loading

overvenus Feb 26, 2024 •

edited

Loading