You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Weiqing Yang (Jira)" <ji...@apache.org> on 2019/11/04 18:49:00 UTC
[jira] [Created] (SAMZA-2363) Rock DB failed to create hard link on
Azure file
Weiqing Yang created SAMZA-2363:
-----------------------------------
Summary: Rock DB failed to create hard link on Azure file
Key: SAMZA-2363
URL: https://issues.apache.org/jira/browse/SAMZA-2363
Project: Samza
Issue Type: Bug
Components: kv, kv-store
Reporter: Weiqing Yang
I was tried to test the implementation ([https://github.com/apache/samza/pull/1197]) of Samza on K8s on AKS (using the latest Samza (master branch)), and got the error below. The job was trying to store logs and the state in Azure file.
(P.S. the implementation works fine with Samza 1.1.)
{code:java}
2019-11-04 00:56:00.558 [main] SamzaContainer [INFO] Entering run loop.
2019-11-04 00:56:00.559 [main] ClusterBasedProcessorLifecycleListener [INFO] Container Started
2019-11-04 00:56:57.106 [kafka-producer-network-thread | kafka_producer-wikipedia_application-1] Metadata [INFO] Cluster ID: YNWIzBLFSa2Lg5wehzo7ZA
2019-11-04 00:57:03.920 [main] KafkaSystemAdmin [INFO] Fetching SSP metadata for: [SystemStreamPartition [kafka, wikipedia-stats-changelog, 7]]
2019-11-04 00:57:03.923 [main] KafkaSystemAdmin [INFO] Fetching SSP metadata for: [SystemStreamPartition [kafka, wikipedia-application-1-window-statsWindow, 7]]
2019-11-04 00:57:04.222 [main] RunLoop [ERROR] Task Partition 7 commit failed
org.rocksdb.RocksDBException: while link file to /tmp/log/wikipedia-application-1/wikipedia-stats/Partition_7-1572829023927.tmp/000013.sst: /tmp/log/wikipedia-application-1/wikipedia-stats/Partition_7/000013.sst: Operation not supported
at org.rocksdb.Checkpoint.createCheckpoint(Native Method)
at org.rocksdb.Checkpoint.createCheckpoint(Checkpoint.java:51)
at org.apache.samza.storage.kv.RocksDbKeyValueStore.checkpoint(RocksDbKeyValueStore.scala:245)
at org.apache.samza.storage.kv.LoggedStore.checkpoint(LoggedStore.scala:125)
at org.apache.samza.storage.kv.SerializedKeyValueStore.checkpoint(SerializedKeyValueStore.scala:170)
at org.apache.samza.storage.kv.CachedStore.checkpoint(CachedStore.scala:297)
at org.apache.samza.storage.kv.NullSafeKeyValueStore.checkpoint(NullSafeKeyValueStore.scala:103)
at org.apache.samza.storage.kv.KeyValueStorageEngine$$anonfun$checkpoint$1.apply(KeyValueStorageEngine.scala:215)
at org.apache.samza.storage.kv.KeyValueStorageEngine$$anonfun$checkpoint$1.apply(KeyValueStorageEngine.scala:212)
at org.apache.samza.util.TimerUtil$class.updateTimer(TimerUtil.scala:37)
at org.apache.samza.storage.kv.KeyValueStorageEngine.updateTimer(KeyValueStorageEngine.scala:37)
at org.apache.samza.storage.kv.KeyValueStorageEngine.checkpoint(KeyValueStorageEngine.scala:212)
at org.apache.samza.storage.TransactionalStateTaskStorageManager$$anonfun$2.apply(TransactionalStateTaskStorageManager.scala:67)
at org.apache.samza.storage.TransactionalStateTaskStorageManager$$anonfun$2.apply(TransactionalStateTaskStorageManager.scala:66)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at org.apache.samza.storage.TransactionalStateTaskStorageManager.checkpoint(TransactionalStateTaskStorageManager.scala:66)
at org.apache.samza.container.TaskInstance.commit(TaskInstance.scala:267)
at org.apache.samza.container.RunLoop$AsyncTaskWorker$5.run(RunLoop.java:547)
at org.apache.samza.container.RunLoop$AsyncTaskWorker.commit(RunLoop.java:566)
at org.apache.samza.container.RunLoop$AsyncTaskWorker.run(RunLoop.java:432)
at org.apache.samza.container.RunLoop$AsyncTaskWorker.access$300(RunLoop.java:357)
at org.apache.samza.container.RunLoop.runTasks(RunLoop.java:244)
at org.apache.samza.container.RunLoop.run(RunLoop.java:176)
at org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:768)
at org.apache.samza.runtime.ContainerLaunchUtil.run(ContainerLaunchUtil.java:139)
at org.apache.samza.runtime.ContainerLaunchUtil.run(ContainerLaunchUtil.java:82)
at org.apache.samza.runtime.LocalContainerRunner.main(LocalContainerRunner.java:80)
2019-11-04 00:57:04.223 [main] RunLoop [ERROR] Caught throwable and stopping run loop
org.rocksdb.RocksDBException: while link file to /tmp/log/wikipedia-application-1/wikipedia-stats/Partition_7-1572829023927.tmp/000013.sst: /tmp/log/wikipedia-application-1/wikipedia-stats/Partition_7/000013.sst: Operation not supported
at org.rocksdb.Checkpoint.createCheckpoint(Native Method)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)