Docs-as-Code Enhanced#
For most of us “Docs-as-Code” mostly means to store the documentation files beside the project sources in git. Also editing the sources in an already used IDE and using the CI system to build it, are 2 important use cases why docs-as-code is chosen to create documentation.
But these features have nothing to do with the documentation content itself. What if the content itself can be treated as code? What if the content / documentation language provides features, which we already know from our programming languages?
Textual and technical representation#
Most documentation exist in two ways out there:
A pure textual representation, readable by human and with some markup to make it nice looking (e.g, MS Word).
A pure technical representation, stored as objects in database and available for humans via graphical user interface only (e.g. a ticket system like JIRA).
The textual representation must be used as it is. No way to easily reorganize it on the fly. No way to easily reuse specific chapters, without the need to copy&paste them. And no way to enhance the content with meta data, to make filtering/searching so much easier.
The technical representation makes all this possible. Add additional data to an object, link it to another object. Filter for it and react programmatically on data changes. All this is possible, with the drawback that you as writer only have objects. You can change the object itself, but there is no way to add information, which are not following the rules of an object. E.g. you can’t add headlines between issue objects. You can have different filters. But you can not create a page with 10 headlines, each filled with a different amount of objects. You don’t have the freedom to organize your content the way a textual representation has.
Objects included documentation#
But what, if you can have both?
The freedom to structure your documentation the way you need it. Plus the possibility to add reusable, linkable, extendable objects on every position of your document?
Use case: Requirements in SW Architecture#
Here an example:
You are responsible for describing the SW architecture of the upcoming project.
And your goal is to not only present UML-models, but also to explain textual why specific decisions were made.
Decisions based on technical and functional requirements. Requirements, which you will need to extend, as your
SW Architecture will tell the SW developers how stuff needs to get implemented.
And a requirement can be best represented as an object, with different kind of information. Able to be linked to other requirements. And accessible all over the documentation to create tables of open requirements or a topic-centric list of requirements. Ready to get exported to external systems or suppliers, without any internal content.
Sphinx & Sphinx-Needs#
This idea got already realised by an extension for the documentation framework Sphinx.
The extension is called Sphinx-Needs and because this page is based on Sphinx, I can present you all features
of Sphinx-Needs right on this page.
Object creation#
So here is an embedded requirement object:
For our overall SW development documentation we need a way to create and reuse objects. We need them for Requirements, Specifications, Test Cases, Test Case Runs and maybe more. |
Click the arrow on the right side of the title to show the meta-data.
The code for this object is embedded inside the normal rst/md documentation code:
Some text before the requirement object.
.. req:: Objects in documentation
:id: REQ_001
:status: done
:tags: objects, documentation, sphinx-needs
:collapse: True
For our overall SW development documentation we need a way to create and reuse objects.
We need them for **Requirements**, **Specifications**, **Test Cases**, **Test Case Runs** and maybe more.
Some text after the requirement object.
Some text before the requirement object.
```{req}Objects in documentation
:id: REQ_001
:status: done
:tags: objects, documentation, sphinx-needs
:collapse: True
For our overall SW development documentation we need a way to create and reuse objects.
We need them for **Requirements**, **Specifications**, **Test Cases**, **Test Case Runs** and maybe more.
```
Some text after the requirement object.
Object Linking#
And for sure objects can be linked to each other:
Objects must be linkable and all incoming and outgoing links shall get documented. This is based on Objects in documentation (REQ_001) |
.. req:: Object linking
:id: REQ_002
:status: done
:tags: objects, documentation, sphinx-needs
:links: REQ_001
Objects must be linkable and all incoming and outgoing links shall get documented.
This is based on :need::REQ_001`
```{req} Object linking
:id: REQ_002
:status: done
:tags: objects, documentation, sphinx-needs
:links: REQ_001
Objects must be linkable and all incoming and outgoing links shall get documented.
This is based on {need}`REQ_001`
```
Tables#
To perform some analysis on objects, we can use needtable.
ID |
Title |
Status |
Type |
Outgoing |
Tags |
|---|---|---|---|---|---|
Objects in documentation |
done |
req |
objects; documentation; sphinx-needs |
||
Object linking |
done |
req |
objects; documentation; sphinx-needs |
.. needtable::
:types: req
:style: table
```{needtable}
:types: req
:style: table
```
And for sure we have access to all objects from the complete documentation (in this case objects created by other blog posts).
The following table is configured to show selected columns and uses the default DataTables for a more “dynamic” view.
ID |
Title |
Status |
Docname |
Section Name |
Incoming |
Outgoing |
|---|---|---|---|---|---|---|
About me |
about |
About me |
||||
Activity object |
posts/2021/process_documents |
Model documentation |
||||
Get documentation |
posts/2021/process_documents |
activities |
||||
Update documentation |
posts/2021/process_documents |
activities |
||||
Upload documentation |
posts/2021/process_documents |
activities |
||||
Build and test doc change |
posts/2021/process_documents |
activities |
||||
Deploy docs |
posts/2021/process_documents |
activities |
||||
Artifact object |
posts/2021/process_documents |
Model documentation |
||||
Sphinx documentation sources |
posts/2021/process_documents |
artifacts |
||||
Sphinx documentation HTML files |
posts/2021/process_documents |
artifacts |
||||
User story |
posts/2021/docs-as-code-enhanced |
User story |
||||
Code language support |
ideas |
Code language support |
||||
Page meta data objects |
index |
Page meta data objects |
||||
Job advertisement for Sphinx-Needs |
posts/2021/job_advertisement_sphinx_needs |
Job advertisement for Sphinx-Needs |
||||
Page meta data in Sphinx |
posts/2021/meta_data_per_page |
Page meta data in Sphinx |
||||
Page meta data in Sphinx |
posts/2021/meta_data_per_page |
Page meta data in Sphinx |
||||
Page meta data in Sphinx |
posts/2021/meta_data_per_page |
Page meta data in Sphinx |
||||
Page meta data in Sphinx |
posts/2021/meta_data_per_page |
Page meta data in Sphinx |
||||
Page meta data in Sphinx |
posts/2021/meta_data_per_page |
Page meta data in Sphinx |
||||
Headlines Example |
posts/2021/meta_data_per_page |
Headlines Example |
||||
MyST |
posts/2021/page_reactivation |
MyST |
||||
Forecast |
posts/2021/meta_data_per_page |
Forecast |
||||
Customer internal |
projects |
Customer internal |
||||
Objects in documentation |
done |
posts/2021/docs-as-code-enhanced |
Object creation |
|||
Object linking |
done |
posts/2021/docs-as-code-enhanced |
Object Linking |
|||
Example |
posts/2021/sphinx_needs_string_links |
Example |
||||
Implement the string2linkfeature |
posts/2021/sphinx_needs_string_links |
Example |
||||
Storage object |
posts/2021/process_documents |
Model documentation |
||||
Git repo github.company.com/team_x/project_y |
posts/2021/process_documents |
storages |
||||
Apache Documentation Webserver |
posts/2021/process_documents |
storages |
||||
Use case example |
posts/mega_sphinx_enterprise/01_planing |
Needed documentation |
||||
PyCharm File Watchers for Sphinx projects |
posts/2021/pycharm_file_watchers |
PyCharm File Watchers for Sphinx projects |
||||
Workflow object |
posts/2021/process_documents |
Model documentation |
||||
Documentation update, build and deploy |
posts/2021/process_documents |
workflows |
.. needtable::
:columns: id,title,status,docname,section_name,incoming,outgoing
```{needtable}
:columns: id,title,status,docname,section_name,incoming,outgoing
```
Flowcharts#
Or maybe present the objects and their connections in a flowchart:
![@startuml
' Nodes definition
node "<size:12>Requirement</size>\n**Objects in**\n**documentation**\n<size:10>REQ_001</size>" as REQ_001 [[../posts/2021/docs-as-code-enhanced.html#REQ_001]] #BFD8D2
node "<size:12>Requirement</size>\n**Object linking**\n<size:10>REQ_002</size>" as REQ_002 [[../posts/2021/docs-as-code-enhanced.html#REQ_002]] #BFD8D2
' Connection definition
REQ_002 --> REQ_001
@enduml](../../_images/plantuml-6cc7209c8833e65b1e1bbd4d9813a3e037bd082b.svg)
.. needflow::
:types: req
```{needflow}
:types: req
```
Export and Import#
All the objects of a documentation can be exported to a json file called needs.json by using the builder
needs: make needs
This is an example of a needs.json file containing 10 needs (including the two from above):
{
"created": "2021-11-19T16:51:30.679372",
"current_version": "",
"project": "danwos",
"versions": {
"": {
"created": "2021-11-19T16:51:30.679346",
"filters": {},
"filters_amount": 0,
"needs": {
"META_DATA": {
"author": "danwos",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": "Explains how to set meta data for a Sphinx page.",
"docname": "posts/2021/meta_data_per_page",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "META_DATA",
"id_complete": "META_DATA",
"id_parent": "META_DATA",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "18.11.2021",
"layout": "",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Page meta data in Sphinx",
"sections": [
"Page meta data in Sphinx"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": null,
"tags": [
"sphinx",
"sphinx-needs",
"meta"
],
"template": null,
"title": "Page meta data in Sphinx",
"type": "metadata",
"type_name": "Meta data",
"updated_at": "",
"url": "",
"user": ""
},
"META_DATA_2": {
"author": "danwos",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": "Explains how to set meta data for a Sphinx page.",
"docname": "posts/2021/meta_data_per_page",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "META_DATA_2",
"id_complete": "META_DATA_2",
"id_parent": "META_DATA_2",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "18.11.2021",
"layout": "meta",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Page meta data in Sphinx",
"sections": [
"Page meta data in Sphinx"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": null,
"tags": [
"sphinx",
"sphinx-needs",
"meta",
"collapse"
],
"template": null,
"title": "Page meta data in Sphinx",
"type": "metadata",
"type_name": "Meta data",
"updated_at": "",
"url": "",
"user": ""
},
"META_DATA_3": {
"author": "danwos",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": ":Title: [[copy(\"section_name\")]]\n:Author: danwos\n:Tags: sphinx, sphinx-needs, meta, collapse, content\n:Last changed: 18.11.2021\n\nExplains how to set meta data for a Sphinx page.",
"docname": "posts/2021/meta_data_per_page",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "META_DATA_3",
"id_complete": "META_DATA_3",
"id_parent": "META_DATA_3",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "18.11.2021",
"layout": "meta",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Page meta data in Sphinx",
"sections": [
"Page meta data in Sphinx"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": null,
"tags": [
"sphinx",
"sphinx-needs",
"meta",
"collapse",
"content"
],
"template": "metadata_template",
"title": "Page meta data in Sphinx",
"type": "metadata",
"type_name": "Meta data",
"updated_at": "",
"url": "",
"user": ""
},
"META_DATA_4": {
"author": "danwos",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": ":Title: [[copy(\"section_name\")]]\n:Author: danwos\n:Tags: sphinx, sphinx-needs, meta, collapse, content, style\n:Last changed: 18.11.2021\n\nExplains how to set meta data for a Sphinx page.",
"docname": "posts/2021/meta_data_per_page",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "META_DATA_4",
"id_complete": "META_DATA_4",
"id_parent": "META_DATA_4",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "18.11.2021",
"layout": "meta",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Page meta data in Sphinx",
"sections": [
"Page meta data in Sphinx"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": "clean",
"tags": [
"sphinx",
"sphinx-needs",
"meta",
"collapse",
"content",
"style"
],
"template": "metadata_template",
"title": "Page meta data in Sphinx",
"type": "metadata",
"type_name": "Meta data",
"updated_at": "",
"url": "",
"user": ""
},
"META_DATA_5": {
"author": "danwos",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": ":Title: [[copy(\"section_name\")]]\n:Author: danwos\n:Tags: sphinx, sphinx-needs, meta, collapse, content, style\n:Last changed: 18.11.2021\n\nExplains how to set meta data for a Sphinx page.",
"docname": "posts/2021/meta_data_per_page",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "META_DATA_5",
"id_complete": "META_DATA_5",
"id_parent": "META_DATA_5",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "18.11.2021",
"layout": "meta",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Page meta data in Sphinx",
"sections": [
"Page meta data in Sphinx"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": "clean",
"tags": [
"sphinx",
"sphinx-needs",
"meta",
"collapse",
"content",
"style"
],
"template": "metadata_template",
"title": "Page meta data in Sphinx",
"type": "metadata",
"type_name": "Meta data",
"updated_at": "",
"url": "",
"user": ""
},
"META_DATA_6": {
"author": "Mr. Nice Guy",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": ":Title: [[copy(\"section_name\")]]\n:Author: Mr. Nice Guy\n:Tags: sphinx, sphinx-needs, meta, collapse, content, style\n:Last changed: 21.21.2020\n\nChapter to explain the usage of ``metadata`` on headline/chapter level.",
"docname": "posts/2021/meta_data_per_page",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "META_DATA_6",
"id_complete": "META_DATA_6",
"id_parent": "META_DATA_6",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "21.21.2020",
"layout": "meta",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Headlines Example",
"sections": [
"Headlines Example",
"Headlines",
"Page meta data in Sphinx"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": "clean",
"tags": [
"sphinx",
"sphinx-needs",
"meta",
"collapse",
"content",
"style"
],
"template": "metadata_template",
"title": "Headlines Example",
"type": "metadata",
"type_name": "Meta data",
"updated_at": "",
"url": "",
"user": ""
},
"REQ_001": {
"author": "",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": "For our overall SW development documentation we need a way to create and reuse objects.\n\nWe need them for **Requirements**, **Specifications**, **Test Cases**, **Test Case Runs** and maybe more.",
"docname": "posts/2021/docs-as-code-enhanced",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "Objects in documentation",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "REQ_001",
"id_complete": "REQ_001",
"id_parent": "REQ_001",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "",
"layout": "",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Object creation",
"sections": [
"Object creation",
"Sphinx & Sphinx-Needs",
"Docs-as-Code Enhanced"
],
"service": "",
"signature": "",
"specific": "",
"status": "done",
"style": null,
"tags": [
"objects",
"documentation",
"sphinx-needs"
],
"template": null,
"title": "Objects in documentation",
"type": "req",
"type_name": "Requirement",
"updated_at": "",
"url": "",
"user": ""
},
"REQ_002": {
"author": "",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": "Objects must be linkable and all incoming and outgoing links shall get documented.\n\nThis is based on {need}`REQ_001`",
"docname": "posts/2021/docs-as-code-enhanced",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "Object linking",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "REQ_002",
"id_complete": "REQ_002",
"id_parent": "REQ_002",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "",
"layout": "",
"links": [
"REQ_001"
],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Object Linking",
"sections": [
"Object Linking",
"Sphinx & Sphinx-Needs",
"Docs-as-Code Enhanced"
],
"service": "",
"signature": "",
"specific": "",
"status": "done",
"style": null,
"tags": [
"objects",
"documentation",
"sphinx-needs"
],
"template": null,
"title": "Object linking",
"type": "req",
"type_name": "Requirement",
"updated_at": "",
"url": "",
"user": ""
},
"SPEC_123": {
"author": "",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": "See above issue on github for a detailed specification",
"docname": "posts/2021/sphinx_needs_string_links",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "Implement the string2linkfeature",
"github": "404",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "SPEC_123",
"id_complete": "SPEC_123",
"id_parent": "SPEC_123",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "",
"layout": "",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Example",
"sections": [
"Example",
"String2Link transformation with Sphinx-Needs"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": null,
"tags": [],
"template": null,
"title": "Implement the string2linkfeature",
"type": "spec",
"type_name": "Specification",
"updated_at": "",
"url": "",
"user": ""
},
"UC_Example": {
"author": "",
"avatar": "",
"closed_at": "",
"completion": "",
"created_at": "",
"description": "Boxes like this will be used to create a link between the technical described solution and a real world\nexample.",
"docname": "posts/mega_sphinx_enterprise/01_planing",
"duration": "",
"external_css": "external_link",
"external_url": null,
"full_title": "Use case example",
"github": "",
"has_dead_links": "",
"has_forbidden_dead_links": "",
"hidden": "",
"id": "UC_Example",
"id_complete": "UC_Example",
"id_parent": "UC_Example",
"id_prefix": "",
"is_external": false,
"is_modified": false,
"is_need": true,
"is_part": false,
"last_changed": "",
"layout": "usecase",
"links": [],
"max_amount": "",
"max_content_lines": "",
"modifications": 0,
"parent_need": null,
"parent_needs": [],
"parent_needs_back": [],
"parts": {},
"post_template": null,
"pre_template": null,
"query": "",
"section_name": "Needed documentation",
"sections": [
"Needed documentation",
"Scenario",
"Sphinx Mega Tutorial for Companies"
],
"service": "",
"signature": "",
"specific": "",
"status": null,
"style": null,
"tags": [],
"template": null,
"title": "Use case example",
"type": "uc",
"type_name": "Use case",
"updated_at": "",
"url": "",
"user": ""
}
},
"needs_amount": 10
}
}
}
And this file and any other file following the same syntax, can be imported by using needimport.
This allows you to extract data from external systems like JIRA, write it to a needs.json compatible file
and import this into your project.
More Features#
Sphinx-Needs provides much more features, like dynamic field values, pie charts, need parts, filters based on Python statements or even Python files. It also has over 40 configuration options, so that it can be adopted to each existing process or tool chain.
See more examples and get the details by reading the Sphinx-Needs Docs.
User story#
Sphinx-Needs is used worldwide by engineering related companies to document their development of software and hardware.
One of my favored project is an ECU development project of a german automotive company. Nearly 2.000 engineers are working on it and they use Sphinx and Sphinx-Needs for documenting their Software Requirements, creating Specifications, planning Test Cases and documenting final Test Case Run Results.
They have multiple Sphinx projects, all linked together using intersphinx and the
external_needs feature of Sphinx-Needs.
Overall they plan to create 130.000 need objects in their documentation.
Their benefit for using a docs-as-code approach are:
More flexibility and full control.
Faster documentation creation.
Higher motivation for developers to write docs.
Team specific analysis and documentations.
No or less license fees for other Requirement tools.
Im/Export of data from external systems.
Single documentation, which can be deployed and safely archived for the next 10-20 years.
So Objects included documentation is come to stay :)
Explains the reason for and features of object orientated documentations. |
Comments
comments powered by Disqus