You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by "masaori335 (via GitHub)" <gi...@apache.org> on 2024/06/04 04:07:24 UTC

[PR] Doc: storage.yaml [trafficserver]

masaori335 opened a new pull request, #11000:
URL: https://github.com/apache/trafficserver/pull/11000

   Draft PR for discuss details of making `volume.config` and `storage.config` into one YAML file.
   
   Most of the docs were written by @SolidWallOfCode last year and I made some changes from his draft. 
   
   Differences are
   
   - Name the config file, I propose `storage.yaml` instead of `cache.yaml`
   - Make the volume id type `integer`, because it's referred from `hosting.config` ( we can change it `string` later, but I'd like to leave it as is for now )
   - Remove percentage support from `span[].size`
   
   I'll open another PR for implementation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11000:
URL: https://github.com/apache/trafficserver/pull/11000#issuecomment-2111435322

   This pull request has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "masaori335 (via GitHub)" <gi...@apache.org>.
masaori335 commented on code in PR #11000:
URL: https://github.com/apache/trafficserver/pull/11000#discussion_r1488989316


##########
doc/admin-guide/files/storage.yaml.en.rst:
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+==============
+storage.yaml
+==============
+
+.. configfile:: storage.yaml
+
+The :file:`storage.yaml` file (by default, located in
+``/usr/local/etc/trafficserver/``) lists all the files, directories, and/or
+hard disk partitions that make up the Traffic Server cache. After you
+modify the :file:`storage.yaml` file the new settings will not be effective until Traffic Server is restarted.
+
+Format
+======
+
+The format of the :file:`storage.yaml` file is a series of lines of the form
+
+.. code-block:: yaml
+
+   cache:           # file level key
+     spans:         #
+       - id:        # identifier / name of the span

Review Comment:
   This is very bike-shedding, but calling this `name` is better than `id`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11000:
URL: https://github.com/apache/trafficserver/pull/11000#issuecomment-2325465009

   This pull request has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #11000: Doc: storage.yaml
URL: https://github.com/apache/trafficserver/pull/11000


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #11000: Doc: storage.yaml
URL: https://github.com/apache/trafficserver/pull/11000


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "masaori335 (via GitHub)" <gi...@apache.org>.
masaori335 commented on code in PR #11000:
URL: https://github.com/apache/trafficserver/pull/11000#discussion_r1483721674


##########
doc/admin-guide/files/storage.yaml.en.rst:
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+==============
+storage.yaml
+==============
+
+.. configfile:: storage.yaml
+
+The :file:`storage.yaml` file (by default, located in
+``/usr/local/etc/trafficserver/``) lists all the files, directories, and/or
+hard disk partitions that make up the Traffic Server cache. After you
+modify the :file:`storage.yaml` file the new settings will not be effective until Traffic Server is restarted.
+
+Format
+======
+
+The format of the :file:`storage.yaml` file is a series of lines of the form
+
+.. code-block:: yaml
+
+   cache:           # file level key
+     spans:         #
+       - id:        # identifier / name of the span
+         path:      # path to storage
+         size:      # size in bytes, required for file system storage, optional for raw device
+         hash_seed: # optional, used to isolate lookup from path changes
+     volumes:       # optional
+       - id:        # identifier [1-255]
+         size:      # optional, size in percentage
+         scheme:    # optional, default to "http"
+         ram-cache: # optional, default to "true"
+         spans:     # optional
+           - use:   # Span identifier
+             size:  # size allocated to this volume
+
+:code:`spans` lists the raw storage used for the cache. :code:`volumes` organizes the storage into locations for
+storing cached objects. This is very similar to operating system partitions and file systems.
+
+For :code:`spans` the keys are
+
++---------------+-------------+-------------------------------------------------------------+
+| Key           | Type        | Meaning                                                     |
++===============+=============+=============================================================+
+| id            | string      | Name of the span.                                           |
++---------------+-------------+-------------------------------------------------------------+
+| path          | string      | File system of the storage. This must be a block device or  |
+|               |             | directory.                                                  |
++---------------+-------------+-------------------------------------------------------------+
+| size          | bytes       | Size in bytes. This is optional for devices but required    |
+|               |             | for directories.                                            |
++---------------+-------------+-------------------------------------------------------------+
+| hash_seed     | string      | Hashing for object location uses a seed to randomize the    |
+|               |             | hash. By default this is the path for the span.             |
++---------------+-------------+-------------------------------------------------------------+
+
+For :code:`volumes` the keys are
+
++---------------+-------------+-------------------------------------------------------------+
+| Key           | Type        | Meaning                                                     |
++===============+=============+=============================================================+
+| id            | integer     | Id of the volume. Range is [1-255]. This id can be referred |
+|               |             | from  :file:`hosting.config`                                |
++---------------+-------------+-------------------------------------------------------------+
+| size          | bytes       | Target size of the entire volume. This can be an absolute   |
+|               | _or_        | number of bytes or a percentage.                            |
+|               | percentage  |                                                             |
++---------------+-------------+-------------------------------------------------------------+
+| scheme        | enumeration | Protocol scheme, defaults to "http". Preserved for future   |
+|               | string      | use.                                                        |
++---------------+-------------+-------------------------------------------------------------+
+| ram-cache     | boolean     | Control of ram caching for this volume. Default is ``true`` |
++---------------+-------------+-------------------------------------------------------------+
+| spans         | list        | Spans that provide storage for this volume. Defaults to     |
+|               |             | all spans.                                                  |
++---------------+-------------+-------------------------------------------------------------+
+
+For :code:`volumes:spans` the keys are
+
++---------------+-------------+-------------------------------------------------------------+
+| Key           | Type        | Meaning                                                     |
++===============+=============+=============================================================+
+| use           | string      | Name of the span to use.                                    |
++---------------+-------------+-------------------------------------------------------------+
+| size          | bytes       | Amount of the span to use. The total across all uses of     |
+|               | _or_        | this specific span must be less than 100% and less than the |
+|               | percentage  | total size of the span.                                     |
++---------------+-------------+-------------------------------------------------------------+
+
+.. important::
+
+   Any change to this files can (and almost always will) invalidate the existing cache in its entirety.
+
+You can use any partition of any size. For best performance:
+
+-  Use raw disk partitions.
+-  For each disk, make all partitions the same size.
+-  Group similar kinds of storage into different volumes. For example
+   split out SSD's or RAM drives into their own volume.
+
+Specify pathnames according to your operating system requirements. See
+the following examples. In the :file:`storage.yaml` file, a formatted or
+raw disk must be at least 128 MB.
+
+When using raw disk or partitions, you should make sure the :ts:cv:`Traffic
+Server user <proxy.config.admin.user_id>` used by the Traffic Server process
+has read and write privileges on the raw disk device or partition. One good
+practice is to make sure the device file is set with 'g+rw' and the Traffic
+Server user is in the group which owns the device file.  However, some
+operating systems have stronger requirements - see the following examples for
+more information.
+
+As with standard ``records.yaml`` integers, human readable prefixes are also
+supported. They include
+
+   - ``K`` Kilobytes (1024 bytes)
+   - ``M`` Megabytes (1024^2 or 1,048,576 bytes)
+   - ``G`` Gigabytes (1024^3 or 1,073,741,824 bytes)
+   - ``T`` Terabytes (1024^4 or 1,099,511,627,776 bytes)
+
+.. _assignment-table:
+
+Storage Allocation
+------------------
+
+Allocation of span storage to volumes is done in stages. Storage is always allocated in multiples of 128 megabytes,
+rounded down.
+
+*  Explicitly sized span storage (:code:`cache:volumes:spans:size`) is allocated to volumes. It is an error if the total allocated is larger than the span size.
+   *  Absolute sizes are allocated first.
+   *  Percentages are allocated from remaining space.
+   *  Remaining storage from spans that are used without an explicit size is divided evenly among the volumes that use the span.
+*  Span storage is allocated to volumes by the :code:`cache:volumes::size` values.
+   *  Absolute sizes are allocated first.
+   *  Percentages are applied to remaining space.
+   *  Remaining storage is divided evenly among volumes without an explicit size.
+
+Assignment Table
+----------------
+
+Each storage element defined in :file:`storage.yaml` is divided in to :term:`stripes <cache stripe>`. The
+assignment table maps from an object URL to a specific stripe. The table is initialized based on a
+pseudo-random process which is seeded by hashing a string for each stripe. This string is composed
+of a base string, an offset (the start of the stripe on the storage element), and the length of the
+stripe. By default the path for the storage is used as the base string. This ensures that each
+stripe has a unique string for the assignment hash. This does make the assignment table very
+sensitive to the path for the storage elements and changing even one can have a cascading effect
+which will effectively clear most of the cache. This can be problem when drives fail and a system
+reboot causes the path names to change.
+
+The :arg:`id` option can be used to create a fixed string that an administrator can use to keep the
+assignment table consistent by maintaining the mapping from physical device to base string even in the presence of hardware changes and failures.
+
+Backwards Compatibility
+-----------------------
+
+In previous versions of |TS| it was possible to have "exclusive" spans which were used by only one volume. This is
+now down by specifying the span in the volume and using a size of "100%". E.g. old configuration like ::
+
+   /dev/disk2 volume=3 # storage.config
+   volume=3 scheme=http size=512 # volume.config
+
+The corresponding configuration would be
+
+.. code-block:: yaml
+
+   cache:
+     spans:
+       - id: disk.2
+         path: /dev/disk2
+     volumes:
+       - id: 1
+         spans:
+           - use: disk.2
+             size: 100%
+
+Because volume sizes that are percentages are computed on span storage not already explicitly allocated, this will
+leave none of "disk.2" for such allocation and therefore "disk.2" will be used only by volume "1". Note this
+configuration is more flexible. If it was useful to have two linear volumes, each using exclusively half of the
+span, this would be
+
+.. code-block:: yaml
+
+   cache:
+     spans:
+       - id: disk.2
+         path: /dev/disk2
+     volumes:
+       - id: 1
+         spans:
+           - use: disk.2
+             size: 50%
+       - id: 2
+         spans:
+           - use: disk.2
+             size: 50%
+
+.. important::
+
+   If a span is explicitly used by any volume its storage will be allocated to only volumes that explicitly use that span.
+
+Examples
+========
+
+The following basic example shows 128 MB of cache storage in the "/big_dir" directory
+
+.. code-block: yaml
+
+   cache:
+     spans:
+       - id: store
+         path: /big_dir
+         size: 134217728
+
+By default a volume uses all spans, therefore a volume uses all of span "store" because there are no other
+volumes. It would be equivalent is using the spans explicitly, e.g.
+
+.. code-block: yaml
+
+   cache:
+     spans:
+       - id: store
+         path: /big_dir
+         size: 134217728
+     volumes:
+       - id: 1
+         size: 100%
+         spans:
+           - id: store
+
+You can use the ``.`` symbol for the current directory. Here is an example for 128 MB of cache storage in the current directory
+
+.. code-block: yaml
+
+   cache:
+     spans:
+       - id: store
+         path: "."
+         size: 134217728
+
+.. note::
+    When using on-filesystem cache disk storage, you can only have one such
+    directory specified. This will be addressed in a future version.
+
+Linux Example
+-------------
+.. note::
+
+   Rather than refer to disk devices like ``/dev/sda``, ``/dev/sdb``, etc.,
+   modern Linux supports `alternative symlinked names for disk devices
+   <https://wiki.archlinux.org/index.php/persistent_block_device_naming#by-id_and_by-path>`_ in the ``/dev/disk``
+   directory structure. As noted for the :ref:`assignment-table` the path used for the disk can effect
+   the cache if it changes. This can be ameliorated in some cases by using one of the alternate paths
+   in via ``/dev/disk``. Note that if the ``by-id`` or ``by-path`` style is used, replacing a failed drive will cause
+   that path to change because the new drive will have a different physical ID or path.
+
+   If this is not sufficient then the :arg:`hash_seed` key should be used to create a more permanent
+   assignment table. An example would be
+
+   .. code-block: yaml
+
+   cache:
+     spans:
+       - id: "span.0"
+         path: "/dev/sde"
+         hash-seed: "cache.disk.0"

Review Comment:
   `hash_seed`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "masaori335 (via GitHub)" <gi...@apache.org>.
masaori335 commented on code in PR #11000:
URL: https://github.com/apache/trafficserver/pull/11000#discussion_r1483721297


##########
doc/admin-guide/files/storage.yaml.en.rst:
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+==============
+storage.yaml
+==============
+
+.. configfile:: storage.yaml
+
+The :file:`storage.yaml` file (by default, located in
+``/usr/local/etc/trafficserver/``) lists all the files, directories, and/or
+hard disk partitions that make up the Traffic Server cache. After you
+modify the :file:`storage.yaml` file the new settings will not be effective until Traffic Server is restarted.
+
+Format
+======
+
+The format of the :file:`storage.yaml` file is a series of lines of the form
+
+.. code-block:: yaml
+
+   cache:           # file level key
+     spans:         #
+       - id:        # identifier / name of the span
+         path:      # path to storage
+         size:      # size in bytes, required for file system storage, optional for raw device
+         hash_seed: # optional, used to isolate lookup from path changes
+     volumes:       # optional
+       - id:        # identifier [1-255]
+         size:      # optional, size in percentage
+         scheme:    # optional, default to "http"
+         ram-cache: # optional, default to "true"

Review Comment:
   Should be `ram_cache` to follow the naming convention.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "brbzull0 (via GitHub)" <gi...@apache.org>.
brbzull0 commented on code in PR #11000:
URL: https://github.com/apache/trafficserver/pull/11000#discussion_r1625918031


##########
doc/admin-guide/files/storage.yaml.en.rst:
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+==============
+storage.yaml
+==============
+
+.. configfile:: storage.yaml
+
+The :file:`storage.yaml` file (by default, located in
+``/usr/local/etc/trafficserver/``) lists all the files, directories, and/or
+hard disk partitions that make up the Traffic Server cache. After you
+modify the :file:`storage.yaml` file the new settings will not be effective until Traffic Server is restarted.
+
+Format
+======
+
+The format of the :file:`storage.yaml` file is a series of lines of the form
+
+.. code-block:: yaml
+
+   cache:           # file level key
+     spans:         #
+       - id:        # identifier / name of the span

Review Comment:
   Probably better and you can differentiate it from `volume:id`, which is a different object I know but a fast reader can confuse. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Doc: storage.yaml [trafficserver]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #11000: Doc: storage.yaml
URL: https://github.com/apache/trafficserver/pull/11000


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org