20200406のTensorFlowに関する記事は2件です。

投稿日：2020-04-06T23:58:17+09:00

ゼロから目指すデータサイエンティスト

TensorFlow - keras でやるMNIST

はじめに

プログラミング初心者なんで、自分が理解するための記録として書いていきたいと思います。
まず、今回やるのはMNISTを使った学習です。

MNISTデータ

MNISTデータは、手書きの数字が書かれた白黒画像（28×28ピクセル）です。
学習データ（60,000枚）、テストデータ（10,000枚）のデータセットです。

データの読み込み

MNISTデータを読み込みます。
データはタプル型で代入されます。
from keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

print(type(mnist.load_data()))
#class <'tuple'>
データの前処理

・入力データの形を変換

入力データを
・学習データ　：60000×784
・テストデータ：10000×784
の形に変換します。
x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
一枚のデータを横一列に並べて、それを縦に60,000（10,000）枚並べた感じになるのかな??
図のイメージ。実際には二次元配列になってると思うけど、、、

・入力データの正規化

学習精度を高めるために、入力データを0〜1の間になるように正規化を行います。
データはfloat型に変換します。
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255
厳密には以下の式で正規化を行っています。
X：入力データ(0〜255)
M：正規化後の最大値1
m：正規化後の最小値0
Y = \frac{X - X_{min}}{X_{max} - X_{min}} (M - m) + m
　

・教師データをダミー変数に変換（one_hot_encoding）

教師データをダミー変数に変換します。
二つ目の引数に値を渡すと、0からnb_classesまでの要素で変換するらしい。
y_train = to_categorical(y_train, 10)
y_test = to_categorical(y_test, 10)

#to_categorical(y, nb_classes = None)
#4 = [0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
モデルの構築

Sequentialモデルで学習をします。
層を追加する時は、addメソットでどんどん追加していくと見易くなります。

Denseの最初の引数はユニット数、activationは活性化関数です。
活性化関数は他にもたくさんあります。
活性化関数は各層でどのように学習して次の層に伝えるを決める結構重要なもの
なのかなって認識しています。

input_dimは隠れ層の一層目に書くもので、入力のユニット数なのかな??

ちなみに、ReLUは0以下の値だと0で、1以上だとそのままの値を出力するような関数です。

softmaxは、出力層で使われます。
使い所としては、分類で識別率を出したいときに使います。
だから、回帰とかの時には使わない認識です。
model = Sequential()
model.add(Dense(64, activation='relu', input_dim=784))
model.add(Dense(10, activation='softmax'))
モデルの最適化

optimizerはRMSpropというものを使います。
optimizerは損失関数を最小にするような重みを算出するアルゴリズムで
その手法は他にもたくさんあります。
ですが、どの手法が優れているのかについてはしっかりとたエビデンスがまだ出ていないらしいです。
ただ、RMSpropはパラメーターが多く、最適なパラメーターを設定しないと他の手法に劣るらしい。
（ここらへんについてはもっと勉強します、、、）

lossは損失関数です。損失関数は、予測と実測値とのズレを評価します。
回帰と分類で、用いる関数が異なります。

metricsは評価関数です。損失関数と似ていて、予測値と実測値とのズレを評価します。
戻り値には、評価した値の平均を返します。
model.compile(
                optimizer='rmsprop',
                loss='categorical_crossentropy',
                metrics=['accuracy']
            )
学習

ここでは、ミニバッチ学習（学習データの半分以下のバッチサイズで学習）をしていきます。
バッチサイズは1グループに対する個数のことです。つまり、バッチサイズが100なら、
600グループができるということになります。ちなみに、グループのことはバッチ数といいます。
グループを分けて学習を行うことで、一部のデータに対してのみ最適化してしまうのを
防ぐ役割をします。そのため、データの動きが激しい場合には、バッチサイズを大きく設定し、
あまり変動がないようなデータにはバッチサイズを小さくするのが望ましいようです。
損失関数の更新はバッチ数毎に行います。（多分、、、）

エポック数は学習回数です。全てのバッチ数を学習し終えると、データをシャッフルします。
この工程がエポック１となります。
model.fit(
            x_train, y_train,
            batch_size=100,
            epochs=12,
            verbose=1
        )
モデルの評価

97.4%の識別率が出ました。
qiita.rb
score = model.evaluate(x_test, y_test)
print(score[0])
print(score[1])

#output
#0.0918885813294677
#0.9739000201225281
終わりに

次回は、boston_housingという地価データがあるらしいので、
回帰分析をして予測して見たいと思います。
あと、出力結果の可視化もやっていきたいと思います。

　 Tweet
Qiitaで続きを読む

投稿日：2020-04-06T19:29:09+09:00

WindowsでTensorFlow liteをビルドする

はじめに

　こういう事で、PoseNetをデスクトップ環境で使いたいとなったけど、いろいろ大変だったのでここに書きます。PoseNetの移植については書きません。タイトルのことのみ書きます。もし、PoseNetを移植したければ、以下のページを参考にしてください。
・Posenet (TensorFlow Lite) のヒートマップをOpenGLESで可視化

Bazelをインストール

　公式ページを参考にインストールしていきます。必須なものは以下です。

　ただし、場合によっては Visual C++ Redistributable for Visual Studio 2015 はすでにインストールされているかもしれません。
　適当なところにbazelを解凍し、解凍したフォルダにパスを通します。そして、コマンドプロンプトで以下のようなコマンドを打ち、こんな感じで返ってきたら成功です。

C:>bazel -help
                                                           [bazel release 2.2.0]
Usage: bazel <command> <options> ...

Available commands:
  analyze-profile     Analyzes build profile data.
  aquery              Analyzes the given targets and queries the action graph.
  build               Builds the specified targets.
  canonicalize-flags  Canonicalizes a list of bazel options.
  clean               Removes output files and optionally stops the server.
  coverage            Generates code coverage report for specified test targets.
  cquery              Loads, analyzes, and queries the specified targets w/ configurations.
  dump                Dumps the internal state of the bazel server process.
  fetch               Fetches external repositories that are prerequisites to the targets.
  help                Prints help for commands, or the index.
  info                Displays runtime info about the bazel server.
  license             Prints the license of this software.
  mobile-install      Installs targets to mobile devices.
  print_action        Prints the command line args for compiling a file.
  query               Executes a dependency graph query.
  run                 Runs the specified target.
  shutdown            Stops the bazel server.
  sync                Syncs all repositories specified in the workspace file
  test                Builds and runs the specified test targets.
  version             Prints version information for bazel.

Getting more help:
  bazel help <command>
                   Prints help and options for <command>.
  bazel help startup_options
                   Options for the JVM hosting bazel.
  bazel help target-syntax
                   Explains the syntax for specifying targets.
  bazel help info-keys
                   Displays a list of keys used by the info command.

TensorFlowをクローンする

　ここは特に言うことありません。Githubからクローンしましょう。

ビルド

　それでは、tensorflow/tensorflow/lite/c/のReadmeに書いてある通りビルドします。といっても、このままでは足りないのでエラーごとに解決法を書いていきます。(ユーザー名は_で隠してます。)

その1（git）

 - D:/program/git/tensolflow/WORKSPACE:37:1
ERROR: An error occurred during the fetch of repository 'io_bazel_rules_docker':
   Traceback (most recent call last):
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git.bzl", line 177
                _clone_or_update(ctx)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git.bzl", line 36, in _clone_or_update
                git_repo(ctx, directory)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 91, in git_repo
                _update(ctx, git_repo)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 101, in _update
                init(ctx, git_repo)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 115, in init
                _error(ctx.name, cl, st.stderr)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 181, in _error
                fail(<1 more arguments>)
error running 'git init C:/users/_/_bazel__/zpebvpgv/external/io_bazel_rules_docker' while working with @io_bazel_rules_docker:
java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("git" init C:/users/_/_bazel__/zpebvpgv/external/io_bazel_rules_docker): ??????????????????
 (error: 2)
ERROR: no such package '@io_bazel_rules_docker//repositories': Traceback (most recent call last):
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git.bzl", line 177
                _clone_or_update(ctx)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git.bzl", line 36, in _clone_or_update
                git_repo(ctx, directory)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 91, in git_repo
                _update(ctx, git_repo)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 101, in _update
                init(ctx, git_repo)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 115, in init
                _error(ctx.name, cl, st.stderr)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 181, in _error
                fail(<1 more arguments>)
error running 'git init C:/users/_/_bazel__/zpebvpgv/external/io_bazel_rules_docker' while working with @io_bazel_rules_docker:
java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("git" init C:/users/_/_bazel__/zpebvpgv/external/io_bazel_rules_docker): ??????????????????
 (error: 2)
ERROR: no such package '@io_bazel_rules_docker//repositories': Traceback (most recent call last):
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git.bzl", line 177
                _clone_or_update(ctx)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git.bzl", line 36, in _clone_or_update
                git_repo(ctx, directory)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 91, in git_repo
                _update(ctx, git_repo)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 101, in _update
                init(ctx, git_repo)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 115, in init
                _error(ctx.name, cl, st.stderr)
        File "C:/users/_/_bazel__/zpebvpgv/external/bazel_tools/tools/build_defs/repo/git_worker.bzl", line 181, in _error
                fail(<1 more arguments>)
error running 'git init C:/users/_/_bazel__/zpebvpgv/external/io_bazel_rules_docker' while working with @io_bazel_rules_docker:
java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("git" init C:/users/_/_bazel__/zpebvpgv/external/io_bazel_rules_docker): ??????????????????

原因

　MSYS2を導入していない。または、MSYS2でgit等を使えるようにしていない。

解決法

　以下のページからMSYS2を導入する。導入の際、セキュリティソフトに引っかかる場合があるので注意。
　インストールをしたら、MSYSを起動し、以下のコマンドでアップデートをする。

pacman -Su

　そしたら、落とせと言われるので×ボタンで落とし、また起動する。そして、以下のコマンドを打ちgit等を使えるようにする。

pacman -S zip unzip patch diffutils git

　そして、コマンドプロンプトからMSYS2のコマンドを使えるようにするため、C:\msys64\usr\binまでパスを通す。

その2（Python was not found）

ERROR: An error occurred during the fetch of repository 'local_config_python':
   Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 210, in _create_local_python_repository
                _get_python_lib(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 130, in _get_python_lib
                execute(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640
Internal error thrown during build. Printing stack trace: java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 210, in _create_local_python_repository
                _get_python_lib(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 130, in _get_python_lib
                execute(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640
, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

INFO: Elapsed time: 36.164s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (11 packages loaded, 13 targets configured)
Internal error thrown during build. Printing stack trace: java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 210, in _create_local_python_repository
                _get_python_lib(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 130, in _get_python_lib
                execute(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640
, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 210, in _create_local_python_repository
                _get_python_lib(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 130, in _get_python_lib
                execute(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640
, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

原因

　Microsoft StoreのPythonがない。Visual Studioとかはダメです。

解決法

　Microsoft StoreでPythonをインストールする。

その3(unicodeescape)

FAILED: Build did NOT complete successfully (11 packages loaded, 13 targets configured)
Internal error thrown during build. Printing stack trace: java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 210, in _create_local_python_repository
                _get_python_lib(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 130, in _get_python_lib
                execute(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
File "<string>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 210, in _create_local_python_repository
                _get_python_lib(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 130, in _get_python_lib
                execute(repository_ctx, <1 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
File "<string>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

原因

　C:\Userとかの\Uがエスケープシーケンスとみなされる。

解決法

　該当箇所を\\Uや/U等に置き換える。今回の例では、D:/program/git/tensolflow/third_party/py/python_configure.bzlL208のpython_binという変数が原因なので、以下のように書き換えます。

def _create_local_python_repository(repository_ctx):
    """Creates the repository containing files set up to build with Python."""

    # Resolve all labels before doing any real work. Resolving causes the
    # function to be restarted with all previous state being lost. This
    # can easily lead to a O(n^2) runtime in the number of labels.
    build_tpl = repository_ctx.path(Label("//third_party/py:BUILD.tpl"))

    python_bin = get_python_bin(repository_ctx).replace('\\', '/')
    _check_python_bin(repository_ctx, python_bin)
    python_lib = _get_python_lib(repository_ctx, python_bin)

その4（No module）

FAILED: Build did NOT complete successfully (11 packages loaded, 13 targets configured)
Internal error thrown during build. Printing stack trace: java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 213, in _create_local_python_repository
                _get_numpy_include(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 187, in _get_numpy_include
                execute(repository_ctx, <3 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Problem getting numpy include path.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
Is numpy installed?, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
java.lang.IllegalStateException: //tensorflow/lite/c:tensorflowlite_c BuildConfigurationValue.Key[b1726b611bdc9a8863c0c9b5691f287791485d5c661e61bcfe9aef1d5e783a56] false -> ErrorInfo{exception=com.google.devtools.build.lib.syntax.EvalExceptionWithStackTrace: Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 263
                _create_local_python_repository(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 213, in _create_local_python_repository
                _get_numpy_include(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/py/python_configure.bzl", line 187, in _get_numpy_include
                execute(repository_ctx, <3 more arguments>)
        File "D:/program/git/tensolflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Problem getting numpy include path.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
Is numpy installed?, rootCauses={REPOSITORY_DIRECTORY:@local_config_python}, cycles=[], isCatastrophic=false, rootCauseOfException=REPOSITORY_DIRECTORY:@local_config_python, isDirectlyTransient=false, isTransitivelyTransient=true}
        at com.google.common.base.Preconditions.checkState(Preconditions.java:823)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.assertSaneAnalysisError(SkyframeBuildView.java:760)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.processErrors(SkyframeBuildView.java:612)
        at com.google.devtools.build.lib.skyframe.SkyframeBuildView.configureTargets(SkyframeBuildView.java:451)
        at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:404)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:213)
        at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:124)
        at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:144)
        at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:288)
        at com.google.devtools.build.lib.runtime.commands.BuildCommand.exec(BuildCommand.java:94)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:564)
        at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:208)
        at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:603)
        at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$2(GrpcServerImpl.java:659)
        at io.grpc.Context$1.run(Context.java:595)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
FAILED: Build did NOT complete successfully (11 packages loaded, 13 targets configured)

原因

　Microsoft Storeのpythonに該当のパッケージがないのが原因です。

解決法

　pipでインストールすればいいのですが、ほかの環境があるとそちらが優先されることが多いため、直接インストールされているところまで行ってpipするのが安全です。なお、私の場合はC:\Users\_\AppData\Local\Microsoft\WindowsApps\にありました。

その5(python path)

ERROR: An error occurred during the fetch of repository 'local_config_git':
   Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 54
                repository_ctx.execute(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 55, in repository_ctx.execute
                _get_python_bin(repository_ctx)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 27, in _get_python_bin
                _fail(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 14, in _fail
                fail(<1 more arguments>)
Git Configuration Error: Cannot find python in PATH, please make sure python is installed and add its directory in PATH, or --define PYTHON_BIN_PATH='/something/else'.
PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\dotnet\;C:\Bazel;C:\msys64\usr\bin;C:\Users\_\AppData\Local\Microsoft\WindowsApps;C:\Users\_\AppData\Local\Programs\Microsoft VS Code\bin;D:\BrainWave\Graphviz\bin;C:\Program Files (x86)\FAHClient;C:\Users\_\.dotnet\tools
INFO: Call stack for the definition of repository 'com_google_absl' which is a tf_http_archive (rule definition at D:/program/git/tensolflow/third_party/repo.bzl:134:19):
 - <builtin>
 - D:/program/git/tensolflow/tensorflow/workspace.bzl:188:5
 - D:/program/git/tensolflow/WORKSPACE:19:1
ERROR: D:/program/git/tensolflow/tensorflow/core/util/BUILD:349:1: //tensorflow/core/util:version_info_gen depends on @local_config_git//:gen/spec.json in repository @local_config_git which failed to fetch. no such package '@local_config_git//': Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 54
                repository_ctx.execute(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 55, in repository_ctx.execute
                _get_python_bin(repository_ctx)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 27, in _get_python_bin
                _fail(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 14, in _fail
                fail(<1 more arguments>)
Git Configuration Error: Cannot find python in PATH, please make sure python is installed and add its directory in PATH, or --define PYTHON_BIN_PATH='/something/else'.
PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\dotnet\;C:\Bazel;C:\msys64\usr\bin;C:\Users\_\AppData\Local\Microsoft\WindowsApps;C:\Users\_\AppData\Local\Programs\Microsoft VS Code\bin;D:\BrainWave\Graphviz\bin;C:\Program Files (x86)\FAHClient;C:\Users\_\.dotnet\tools
ERROR: D:/program/git/tensolflow/tensorflow/core/util/BUILD:349:1: //tensorflow/core/util:version_info_gen depends on @local_config_git//:gen/head in repository @local_config_git which failed to fetch. no such package '@local_config_git//': Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 54
                repository_ctx.execute(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 55, in repository_ctx.execute
                _get_python_bin(repository_ctx)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 27, in _get_python_bin
                _fail(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 14, in _fail
                fail(<1 more arguments>)
Git Configuration Error: Cannot find python in PATH, please make sure python is installed and add its directory in PATH, or --define PYTHON_BIN_PATH='/something/else'.
PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\dotnet\;C:\Bazel;C:\msys64\usr\bin;C:\Users\_\AppData\Local\Microsoft\WindowsApps;C:\Users\_\AppData\Local\Programs\Microsoft VS Code\bin;D:\BrainWave\Graphviz\bin;C:\Program Files (x86)\FAHClient;C:\Users\_\.dotnet\tools
ERROR: D:/program/git/tensolflow/tensorflow/core/util/BUILD:349:1: //tensorflow/core/util:version_info_gen depends on @local_config_git//:gen/branch_ref in repository @local_config_git which failed to fetch. no such package '@local_config_git//': Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 54
                repository_ctx.execute(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 55, in repository_ctx.execute
                _get_python_bin(repository_ctx)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 27, in _get_python_bin
                _fail(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 14, in _fail
                fail(<1 more arguments>)
Git Configuration Error: Cannot find python in PATH, please make sure python is installed and add its directory in PATH, or --define PYTHON_BIN_PATH='/something/else'.
PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\dotnet\;C:\Bazel;C:\msys64\usr\bin;C:\Users\_\AppData\Local\Microsoft\WindowsApps;C:\Users\_\AppData\Local\Programs\Microsoft VS Code\bin;D:\BrainWave\Graphviz\bin;C:\Program Files (x86)\FAHClient;C:\Users\_\.dotnet\tools
ERROR: Analysis of target '//tensorflow/lite/c:tensorflowlite_c' failed; build aborted: no such package '@local_config_git//': Traceback (most recent call last):
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 54
                repository_ctx.execute(<2 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 55, in repository_ctx.execute
                _get_python_bin(repository_ctx)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 27, in _get_python_bin
                _fail(<1 more arguments>)
        File "D:/program/git/tensolflow/third_party/git/git_configure.bzl", line 14, in _fail
                fail(<1 more arguments>)
Git Configuration Error: Cannot find python in PATH, please make sure python is installed and add its directory in PATH, or --define PYTHON_BIN_PATH='/something/else'.
PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\dotnet\;C:\Bazel;C:\msys64\usr\bin;C:\Users\_\AppData\Local\Microsoft\WindowsApps;C:\Users\_\AppData\Local\Programs\Microsoft VS Code\bin;D:\BrainWave\Graphviz\bin;C:\Program Files (x86)\FAHClient;C:\Users\_\.dotnet\tools
INFO: Elapsed time: 8.178s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (37 packages loaded, 261 targets configured)
    Fetching @farmhash_archive; fetching
    Fetching ...rchive; Extracting C:/users/_/_bazel__/zpebvpgv/external/farmhash_archive/816a4ae622e964763ca0\
862d9dbd19324a1eaf45.tar.gz

原因

　MicrosoftのpythonへのパスであるPYTHON_BIN_PATHが設定されていません。

解決法

　環境変数で、PYTHON_BIN_PATHが設定します。

さいごに

　私のであったエラーは以上です。ほんとはtensorflow liteと自分のものを分けないといけないけど、VCでリンクさせるのがなんかできなかったからapi.h、api.cに直接書いてしまった...

　 Tweet
Qiitaで続きを読む

20200406のTensorFlowに関する記事は2件です。

ゼロから目指すデータサイエンティスト

TensorFlow - keras でやるMNIST

はじめに

MNISTデータ

データの読み込み

データの前処理

・入力データの形を変換

・入力データの正規化

・教師データをダミー変数に変換（one_hot_encoding）

モデルの構築

モデルの最適化

学習

モデルの評価

終わりに

WindowsでTensorFlow liteをビルドする

はじめに

Bazelをインストール

TensorFlowをクローンする

ビルド

その1（git）

原因

解決法

その2（Python was not found）

原因

解決法

その3(unicodeescape)

原因

解決法

その4（No module）

原因

解決法

その5(python path)

原因

解決法

さいごに

日付別

タグ一覧