Adding Custom RR Graph Builder to OpenFPGA

.gitmodules

@@ -16,6 +16,9 @@
 [submodule "libs/EXTERNAL/yosys-slang"]
 	path = libs/EXTERNAL/yosys-slang
 	url = https://github.com/povik/yosys-slang.git
+[submodule "libs/EXTERNAL/yaml-cpp"]
+	path = libs/EXTERNAL/yaml-cpp
+	url = https://github.com/jbeder/yaml-cpp.git
 [submodule "libs/EXTERNAL/libsdcparse"]
 	path = libs/EXTERNAL/libsdcparse
 	url = https://github.com/verilog-to-routing/libsdcparse.git

doc/src/vpr/command_line_usage.rst

@@ -1,3 +1,5 @@
+.. _vpr_command_line_usage:
+
 Command-line Options
 ====================
 .. program:: vpr

doc/src/vpr/custom_rr_graph.rst

@@ -0,0 +1,187 @@
+.. _custom_rr_graph_generator:
+
+Custom RR Graph
+===============
+
+For users who want more control over how connections are made in the routing resource graph, VPR provides a way to describe them through a Custom RR Graph (CRR) generator.
+
+Currently, the CRR Generator is only based on the tileable RR Graph. Support for the default VPR RR Graph Generator is in progress. To generate a CRR, the following files are required:
+
+* Switch block map file
+* Switch block template files
+
+Switch Block Map File
+~~~~~~~~~~~~~~~~~~~~~~
+
+This is a YAML file that specifies which switch block template should be used for each tile. The mapping format supports the following patterns:
+
+* ``SB_1__1_: [sb_template.csv]`` - Tile at location (1, 1) uses switch block template ``sb_template.csv``
+* ``SB_1__*_: [sb_template.csv]`` - All tiles with x coordinate 1 use switch block template ``sb_template.csv``
+* ``SB_[7,20]__[2:32:3]_: [sb_template.csv]`` - Tiles with x equal to 7 or 20, and y coordinates from 2 to 32 (inclusive) with step 3 use switch block template ``sb_template.csv``
+
+Below is an example of a switch block map file where there is no switch block at the perimeter, and DSP blocks are placed from block 2 to 10 (inclusive) with a step of 2. For all other locations, the same switch block template is used.
+
+.. code-block:: yaml
+
+   SB_MAPS:
+   SB_0__0_: null
+   SB_1__: sb_perimeter.csv
+   SB1: sb_perimeter.csv
+   SB[2:10:2]: sb_dsp.csv
+   SB*_: sb_main.csv
+
+**Important:** The order in which patterns are defined matters, as the first matching pattern is used.
+
+Switch Block Template Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Terminology
+-----------
+
+Before describing the template files, let's define some key terminology:
+
+**Lane:** A lane is a group of routing wires that shuffle only within the boundaries of their own lane region as they propagate across tiles.
+
+**Tap:** A switch block location where a wire passes through and can have fan-out connections.
+
+See the figure below for an illustration:
+
+.. figure:: lane_and_tap.png
+   :alt: Lane and Tap
+   :align: center
+
+   In the figure above, the taps for the red wire are shown, and the lanes are separated by dotted lines. Note that the figure is simplified for illustration purposes.
+
+In the actual RR Graph, when a wire passes through a switch block, its track number changes. Below is a more realistic example:
+
+.. figure:: lane_and_tap_realistic.png
+   :alt: Lane and Tap Realistic
+   :align: center
+
+   A more realistic example of a lane and a tap. Each box contains a lane.
+
+Template File Format
+--------------------
+
+There should be a directory containing the pattern files specified in the switch block maps file. Each template is a CSV file with the following format:
+
+* **Rows** represent source nodes
+* **Columns** represent sink nodes
+* An **'x' mark** at an intersection indicates that the source and sink nodes are connected. The delay for that connections is retrieved from the sitch type specified in the architecture file.
+* A **number** at an intersection indicates that the nodes are connected with the switch delay specified by that number
+
+**Note:** The pattern currently only supports uni-directional segments. Therefore, wires can only be driven from their starting point.
+
+Row Headers (Source Nodes)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Each row has four header columns describing the source node:
+
+1. **Column 1 - Direction:** The side from which the source node is entering the switch block (e.g., 'left', 'right', 'top', 'bottom')
+2. **Column 2 - Segment Type:** The segment length to which the source node belongs
+3. **Column 3 - Lane Number:** The lane to which the source node belongs (see terminology above)
+4. **Column 4 - Tap Number:** Which tap of the source node is at this switch block
+
+Column Headers (Sink Nodes)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Each column has header rows describing the sink node:
+
+1. **Row 1 - Direction:** The side from which the sink node is exiting the switch block
+2. **Row 2 - Segment Type:** The segment length to which the sink node belongs
+3. **Row 3 - Fan-in:** The number of fan-ins to the sink node (optional)
+4. **Row 4 - Lane Number:** The lane to which the sink node belongs
+
+*Note:* Given that currently we only support unidirectional segments, the tap is not specified because a wire can only be driven from its starting point.
+
+Example
+^^^^^^^
+
+Consider an architecture with a channel width of ``160`` containing only wire
+segments of length ``4`` (L4).
+
+The number of **rows** in the template file is computed as:
+
+- ``4`` (number of sides) × ``160/2`` (number of tracks in one direction)
+  = ``320`` rows.
+
+On each side, we have:
+
+- ``20`` lanes (``80 / 4``), and
+- each lane requires ``4`` rows (Lane width is equal to the segment length).
+
+The number of **columns** is:
+
+- ``4`` (number of sides) × ``160/2`` (number of tracks in one direction) ÷ ``4`` (starting tracks per lane)
+  = ``80`` columns.
+
+**Note:** The reason for dividing by 4 is that only the first track of each lane can be driven from each switch location.
+
+----
+
+To illustrate how a particular cell in the template file is interpreted, assume
+the following row and column specifications:
+
+**Row specification:**
+
+- Direction: ``left``
+- Segment type: ``l4``
+- Lane number: ``2``
+- Tap number: ``3``
+
+**Column specification:**
+
+- Direction: ``right``
+- Segment type: ``l4``
+- Lane number: ``2``
+
+Assuming ``l4`` is the only segment type, the row specification corresponds to a
+source wire entering the switch block from the *left* side. The starting PTC
+(track) number for lane 2 is:
+
+.. code-block:: none
+
+   (2 - 1) * 4 * 2 = 8
+
+Thus, the PTC sequence for this L4 wire is (it increases by 2 since the odd PTC numbers are used for wires coming from the other side):
+
+.. code-block:: none
+
+   8, 10, 12, 14
+
+The row refers to the node associated with the **third tap**, meaning the
+source node corresponds to the PTC value ``12``. With the starting and ending of the track,
+and the PTC values, VPR can determine the source **Node ID**.
+
+For the sink node, the column specification describes a wire exiting the switch
+block on the *right* side, using the same PTC sequence:
+
+.. code-block:: none
+
+   8, 10, 12, 14
+
+Using this sequence, VPR determines the sink **Node ID**.
+
+After computing both Node IDs, the tool inserts the switch defined in the
+template file, with the associated delay value, into the RR Graph.
+
+----
+
+Each switch block template file must contain the number of rows and columns
+calculated in the previous section. After creating a correctly sized template
+file, populate it by placing switch delay values in the cells representing valid
+connections between source and sink nodes, or if you want to use the switches defined
+in the architecture file, you can mark the connection with an 'x'.
+
+Once the switch block map file and template files are created, include the
+following arguments in the VPR command line:
+
+- ``--sb_maps <switch_block_map_file>``
+- ``--sb_templates <switch_block_template_directory>``
+- ``--sb_count_dir <switch_block_count_directory>`` (optional):  
+  If provided, VPR generates a CSV file for each switch block template,
+  showing how many times each switch specified in the template is used in the
+  final routing results.
+
+For additional arguments, refer to the command-line usage section in
+:ref:`vpr_command_line_usage`.

doc/src/vpr/index.rst

@@ -60,3 +60,4 @@ The purpose of VPR is to make the packing, placement, and routing stages of the
 
    file_formats
    debug_aids
+   custom_rr_graph_generator

libs/EXTERNAL/CMakeLists.txt

@@ -12,6 +12,16 @@ add_subdirectory(libsdcparse)
 add_subdirectory(libblifparse)
 add_subdirectory(libtatum)
 add_subdirectory(libcatch2)
+
+add_subdirectory(yaml-cpp)
+# Treat yaml-cpp headers as system headers to suppress warnings
+if(TARGET yaml-cpp)
+    get_target_property(YAML_INCLUDE_DIRS yaml-cpp INTERFACE_INCLUDE_DIRECTORIES)
+    set_target_properties(yaml-cpp PROPERTIES
+        INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${YAML_INCLUDE_DIRS}"
+    )
+endif()
+
 #add_subdirectory(parmys)
 
 #Proc numbers

libs/libarchfpga/src/physical_types.h

@@ -1799,6 +1799,12 @@ struct t_arch_switch_inf {
     e_power_buffer_type power_buffer_type = POWER_BUFFER_TYPE_AUTO;
     float power_buffer_size = 0.;
 
+    // The template ID of the switch. This is metadata stored for each switch to
+    // simplify certain analyses. For example, when generating the CRR graph, the
+    // template ID is used to determine which switch in the template is used most
+    // or least frequently.
+    std::string template_id = "";
+
     bool intra_tile = false;
 
   public:

libs/librrgraph/src/base/rr_graph_builder.cpp

@@ -176,7 +176,10 @@ void RRGraphBuilder::reorder_nodes(e_rr_node_reorder_algorithm reorder_rr_graph_
     });
 }
 
-void RRGraphBuilder::create_edge_in_cache(RRNodeId src, RRNodeId dest, RRSwitchId edge_switch, bool remapped) {
+void RRGraphBuilder::create_edge_in_cache(RRNodeId src,
+                                          RRNodeId dest,
+                                          RRSwitchId edge_switch,
+                                          bool remapped) {
     edges_to_build_.emplace_back(src, dest, size_t(edge_switch), remapped);
     is_edge_dirty_ = true; // Adding a new edge revokes the flag
     is_incoming_edge_dirty_ = true;

libs/librrgraph/src/base/rr_graph_builder.h

@@ -12,6 +12,7 @@
  *
  */
 
+#include <string>
 #include "rr_graph_storage.h"
 #include "rr_spatial_lookup.h"
 #include "metadata_storage.h"
@@ -289,7 +290,10 @@ class RRGraphBuilder {
 
     /** @brief Add a new edge to the cache of edges to be built 
      *  @note This will not add an edge to storage. You need to call build_edges() after all the edges are cached. */
-    void create_edge_in_cache(RRNodeId src, RRNodeId dest, RRSwitchId edge_switch, bool remapped);
+    void create_edge_in_cache(RRNodeId src,
+                              RRNodeId dest,
+                              RRSwitchId edge_switch,
+                              bool remapped);
 
     /** @brief Add a new edge to the cache of edges to be built 
      *  @note This will not add an edge to storage! You need to call build_edges() after all the edges are cached! */
@@ -332,7 +336,10 @@ class RRGraphBuilder {
      * remap the arch switch id to rr switch id, the edge switch id of this edge shouldn't be changed. For example, when the intra-cluster graph
      * is built and the rr-graph related to global resources are read from a file, this parameter is true since the intra-cluster switches are
      * also listed in rr-graph file. So, we use that list to use the rr switch id instead of passing arch switch id for intra-cluster edges.*/
-    inline void emplace_back_edge(RRNodeId src, RRNodeId dest, short edge_switch, bool remapped) {
+    inline void emplace_back_edge(RRNodeId src,
+                                  RRNodeId dest,
+                                  short edge_switch,
+                                  bool remapped) {
         node_storage_.emplace_back_edge(src, dest, edge_switch, remapped);
     }
     /** @brief Append 1 more RR node to the RR graph. */
@@ -401,6 +408,16 @@ class RRGraphBuilder {
         return node_storage_.count_rr_switches(arch_switch_inf, arch_switch_fanins);
     }
 
+    /**
+     * @brief Unlock storage; required to modify an routing resource graph after edge is read
+     * @note This function is used by OpenFPGA and currently doesn't have any use in VPR code.
+     */
+    inline void unlock_storage() {
+        node_storage_.edges_read_ = false;
+        node_storage_.partitioned_ = false;
+        node_storage_.clear_node_first_edge();
+    }
+
     /** @brief Reserve the lists of nodes, edges, switches etc. to be memory efficient.
      * This function is mainly used to reserve memory space inside RRGraph,
      * when adding a large number of nodes/edge/switches/segments,
@@ -420,7 +437,6 @@ class RRGraphBuilder {
         node_storage_.resize(size);
     }
 
-
     /** @brief This function resize rr_switch to accommodate size RR Switch. */
     inline void resize_switches(size_t size) {
         rr_switch_inf_.resize(size);

libs/librrgraph/src/base/rr_graph_storage.cpp

@@ -18,7 +18,10 @@ void t_rr_graph_storage::reserve_edges(size_t num_edges) {
     edge_remapped_.reserve(num_edges);
 }
 
-void t_rr_graph_storage::emplace_back_edge(RRNodeId src, RRNodeId dest, short edge_switch, bool remapped) {
+void t_rr_graph_storage::emplace_back_edge(RRNodeId src,
+                                           RRNodeId dest,
+                                           short edge_switch,
+                                           bool remapped) {
     // Cannot mutate edges once edges have been read!
     VTR_ASSERT(!edges_read_);
     edge_src_node_.emplace_back(src);

libs/librrgraph/src/base/rr_graph_storage.h

@@ -20,6 +20,7 @@
 #include <numeric>
 #include <optional>
 #include <cstdint>
+#include <string>
 #include <vector>
 
 /* Main structure describing one routing resource node.  Everything in       *
@@ -826,7 +827,10 @@ class t_rr_graph_storage {
      * of the node. Also, the information about switches is fly-weighted and are accessible with IDs. Thus,
      * the number of rr switch types can be higher than the number of arch switch types.
      */
-    void emplace_back_edge(RRNodeId src, RRNodeId dest, short edge_switch, bool remapped);
+    void emplace_back_edge(RRNodeId src,
+                           RRNodeId dest,
+                           short edge_switch,
+                           bool remapped);
 
     /** @brief Adds a batch of edges.*/
     void alloc_and_load_edges(const t_rr_edge_info_set* rr_edges_to_create);

libs/librrgraph/src/base/rr_graph_view.cpp

@@ -61,9 +61,10 @@ std::vector<RREdgeId> RRGraphView::node_non_configurable_in_edges(RRNodeId node)
 
 std::vector<RREdgeId> RRGraphView::find_edges(RRNodeId src_node, RRNodeId des_node) const {
     std::vector<RREdgeId> edge_list;
-    for (auto iedge : node_out_edges(src_node)) {
-        if (edge_sink_node(RREdgeId(iedge)) == des_node) {
-            edge_list.push_back(RREdgeId(iedge));
+    for (auto iedge: node_out_edges(src_node)) {
+        RREdgeId edge = node_storage_.edge_id(src_node, iedge);
+        if (edge_sink_node(edge) == des_node) {
+            edge_list.push_back(edge);
         }
     }
     return edge_list;

libs/librrgraph/src/base/rr_graph_view.h

@@ -63,6 +63,7 @@
  */
 
 #include "metadata_storage.h"
+#include "rr_graph_fwd.h"
 #include "rr_node.h"
 #include "physical_types.h"
 #include "rr_node_types.h"

libs/librrgraph/src/base/rr_node_types.h

@@ -56,6 +56,30 @@ constexpr vtr::array<e_rr_type, const char*, (size_t)e_rr_type::NUM_RR_TYPES> rr
                                                                                                "CHANX", "CHANY", "CHANZ",
                                                                                                "MUX"};
 
+inline e_rr_type get_rr_type(const std::string& type_name) {
+    if (type_name == "SOURCE") {
+        return e_rr_type::SOURCE;
+    } else if (type_name == "SINK") {
+        return e_rr_type::SINK;
+    } else if (type_name == "IPIN") {
+        return e_rr_type::IPIN;
+    } else if (type_name == "OPIN") {
+        return e_rr_type::OPIN;
+    } else if (type_name == "CHANX") {
+        return e_rr_type::CHANX;
+    } else if (type_name == "CHANY") {
+        return e_rr_type::CHANY;
+    } else if (type_name == "CHANZ") {
+        return e_rr_type::CHANZ;
+    } else if (type_name == "MUX") {
+        return e_rr_type::MUX;
+    } else {
+        std::string error_message = "Invalid RR type name: " + type_name;
+        VTR_ASSERT_MSG(false, error_message.c_str());
+        return e_rr_type::NUM_RR_TYPES;
+    }
+}
+
 /**
  * @enum Direction
  * @brief Represents the wire direction for a routing resource node.