Tensorflow hizmeti yeniden başlangıç ​​başlangıcı

Question

this guide'u izledikten sonra yeniden başlatılmış başlangıç ​​modeline hizmet vermeye çalışıyorum (ayrıca, başlangıcının nasıl yeniden değerlendirileceğini açıklayan this guide da görebilirsiniz).

/serving/bazel-bin/tensorflow_serving/example/inception_inference --port=9000 EXPORT_DIR &> inception_log & 

Sunucusu günlük dosyası (inception_log) içerir:

... # Same as in the original script: 
# Set up the pre-trained graph. 
maybe_download_and_extract() 
graph, bottleneck_tensor, jpeg_data_tensor, resized_image_tensor = (create_inception_graph()) 
... # Same as in the original script: 
# Add the new layer that we'll be training. 
(train_step, cross_entropy, bottleneck_input, ground_truth_input, final_tensor) = add_final_training_ops(len(image_lists.keys()), 
             FLAGS.final_tensor_name, 
             bottleneck_tensor) 
... # Added at the end of the original script: 
# Export model 
with graph.as_default(): 
    export_path = sys.argv[-1] 
    print('Exporting trained model to', export_path) 
    saver = tf.train.Saver(sharded=True) 
    model_exporter = exporter.Exporter(saver) 
    signature = exporter.classification_signature(input_tensor=jpeg_data_tensor, scores_tensor=final_tensor) 
    model_exporter.init(sess.graph.as_graph_def(), default_graph_signature=signature) 
    model_exporter.export(export_path, tf.constant(FLAGS.export_version), sess) 
    print('Done exporting!') 

if __name__ == '__main__': 
    tf.app.run() 

Modelimin ihraç sonra sunucu çalışmaya başlar şu şekildedir: Ben modeli ihraç etmeye retrain.py modifiye ettik

I tensorflow_serving/core/basic_manager.cc:190] Using InlineExecutor for BasicManager. 
I tensorflow_serving/example/inception_inference.cc:384] Waiting for models to be loaded... 
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001 
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:129] Attempting to load a SessionBundle from: /tf_files/scope/export/00000001 
I tensorflow_serving/example/inception_inference.cc:384] Waiting for models to be loaded... 
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001 
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:106] Running restore op for SessionBundle 
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:203] Done loading SessionBundle 
I tensorflow_serving/example/inception_inference.cc:350] Running... 
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001 
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001 
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001 
... 

Son olarak, istemciyi çalıştırıyorum ve şu hatayı alıyorum:

/serving/bazel-bin/tensorflow_serving/example/inception_client --server=localhost:9000 --image=TEST_IMG 
D0805 09:10:46.208704633  200 ev_posix.c:101]    Using polling engine: poll 
Traceback (most recent call last): 
    File "/serving/bazel-bin/tensorflow_serving/example/inception_client.runfiles/tensorflow_serving/example/inception_client.py", line 53, in <module> 
    tf.app.run() 
    File "/serving/bazel-bin/tensorflow_serving/example/inception_client.runfiles/external/org_tensorflow/tensorflow/python/platform/app.py", line 30, in run 
    sys.exit(main(sys.argv)) 
    File "/serving/bazel-bin/tensorflow_serving/example/inception_client.runfiles/tensorflow_serving/example/inception_client.py", line 48, in main 
    result = stub.Classify(request, 10.0) # 10 secs timeout 
    File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 300, in __call__ 
    self._request_serializer, self._response_deserializer) 
    File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 198, in _blocking_unary_unary 
    raise _abortion_error(rpc_error_call) 
    grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INTERNAL, details="FetchOutputs node : not found") 
E0805 09:10:47.129263239  200 chttp2_transport.c:1810] close_transport: {"created":"@1470388247.129230608","description":"FD shutdown","file":"src/core/lib/iomgr/ev_poll_posix.c","file_line":427} 

Bu konuyla ilgili herhangi bir tavsiye veya rehberlik büyük ölçüde takdir edilecektir.

Answer 1

Tensorflow web sitesindeki bağlantı, modelimi deneyimimin tamamından sunmanın sadece bir yoludur. modeli hizmet etmek daha iyi bir yol altyapısını sunan tensorflow hepsinden daha onun daha hafif bir ağırlık istemcisi beri yerine Flask ve Kubernetes dan o kadar hizmet etmek olurdu, ama bu sahip olduğunuz hacmi çok büyük değil varsayarak (> 100 QPS) Eğer ancak in-line çözüm için op olur o oranda, o yük ile Flask ve Kubernetes ile Inception kadar hizmet verebilir rağmen.

Bunu bir uzaktan hizmetten sunabilirdiniz ve bu işe yarayabilir, ancak altyapınıza bağlı olarak, bir apache_beam.DoFn aracılığıyla isteklerinizi zorlayan bir akış işinde bu modeli sunabilir ve daha sonra çıktı alabilirsiniz. işinizin dinlediği bir MQ'ya. Bu sadece başka bir çözüm. Bu yardımcı olur umarım.

from __future__ import absolute_import 
from __future__ import division 
from __future__ import print_function 

import logging 
import tensorflow as tf 
import numpy as np 
import apache_beam as beam 


class InferenceFn(beam.DoFn): 

    def __init__(self, model_dict): 
    super(InferenceFn, self).__init__() 
    self.model_dict = model_dict 
    self.graph = None 
    self.create_graph() 


    def create_graph(self): 
    if not tf.gfile.FastGFile(self.model_dict['model_full_path']): 
     self.download_model_file() 
    with tf.Graph().as_default() as graph: 
     with tf.gfile.FastGFile(self.model_dict['model_full_path'], 'rb') as f: 
     graph_def = tf.GraphDef() 
     graph_def.ParseFromString(f.read()) 
     _ = tf.import_graph_def(graph_def, name='') 
    self.graph = graph 

    def start_bundle(self): 
    """Prevents graph object serialization until serving. Required for GCP Serving""" 
    self.create_graph() 

    def process(self, element): 
    """Core Processing Fn for Apache Beam.""" 
    try: 
     with tf.Session(graph=self.graph) as sess: 
     if not tf.gfile.Exists(element): 
      tf.logging.fatal('File does not exist %s', element) 
      raise ReferenceError("Couldnt Find the image {}".format(element)) 
     data = tf.gfile.FastGFile(element, 'rb').read() 
     output_tensor = sess.graph.get_tensor_by_name(self.model_dict['output_tensor_name']) 
     predictions = sess.run(softmax_tensor, {self.model_dict['input_tensor_name']: data}) 
     predictions = np.squeeze(predictions) 
     yield str(predictions) 
    except Exception: 
     logging.error("We hit an error in inference on {}".format(element))