[WIP] Add additional linting to package#10
Conversation
|
@SorooshMani-NOAA I'd like your feedback specifically about how to handle the remaining errors and your thoughts about integrating tests for these in the Github Action testing workflow. |
| # refine_opts.mesh_top1 = True | ||
| # refine_opts.geom_feat = True |
There was a problem hiding this comment.
Please remove extra spaces between # and the commented code following it now that the lines are shifted
| # if out_crs is not None: | ||
| # utils.reproject(jig_remeshed, out_crs) |
There was a problem hiding this comment.
Same as above, please remove extra spaces
| "--contour", | ||
| action="append", | ||
| nargs="+", | ||
| type=float, | ||
| dest="contours", | ||
| metavar="CONTOUR_DEFN", | ||
| default=[], |
There was a problem hiding this comment.
I think it's too much to spread every argument to a newline. Is there any way to control linter not to do this?
| from ocsmesh import Geom, Hfun, JigsawDriver, Raster | ||
| from ocsmesh.features.contour import Contour | ||
| from ocsmesh.geom.shapely import MultiPolygonGeom | ||
| from ocsmesh.hfun.mesh import HfunMesh | ||
| from ocsmesh.features.contour import Contour | ||
| from ocsmesh.mesh.mesh import Mesh | ||
| from ocsmesh.mesh.parsers import sms2dm | ||
| from ocsmesh.utils import msh_t_to_2dm |
There was a problem hiding this comment.
I think it's better if we have a 3rd block of imports specific to the module within this package (ocsmesh) and then organize them separately instead of mixing with other third_party
| raster_path, | ||
| raster_opts, | ||
| zmin, | ||
| zmax, | ||
| join_method, | ||
| driver, | ||
| chunk_size, | ||
| overlap, |
There was a problem hiding this comment.
I'm not sure how I feel about removing 1 level of indentation here. It looks OK, but I don't know if it causes more confusion or not do to aligning with function body
| self, | ||
| geom: Geom, | ||
| hfun: Hfun, | ||
| initial_mesh: bool = False, | ||
| crs: Union[str, CRS] = None, | ||
| verbosity: int = 0, |
There was a problem hiding this comment.
The same issue with indentation aligning with function body
| # if self.opts.hfun_hmin > 0: | ||
| # output_mesh = utils.remesh_small_elements( | ||
| # self.opts, geom_msh_t, output_mesh, hfun_msh_t) |
There was a problem hiding this comment.
Please remove extras spaces after # now that the lines are shifted
| self, | ||
| value, | ||
| upper_bound=np.inf, | ||
| lower_bound=-np.inf, | ||
| value_type: str = "min", | ||
| rate=None, |
| self, | ||
| function=lambda i: i / 2.0, | ||
| upper_bound=np.inf, | ||
| lower_bound=-np.inf, | ||
| value_type: str = "min", | ||
| rate=None, |
There was a problem hiding this comment.
Just marking as same comment!
| pass | ||
|
|
||
| # @abstractmethod | ||
| # @abstractmethod |
There was a problem hiding this comment.
Remove extra spaces after #
| self, | ||
| level0=None, | ||
| level1=None, | ||
| sources=[], | ||
| max_contour_defn: Contour = None, | ||
| shapefile=None, |
There was a problem hiding this comment.
Indentation alignment thing!
| self, | ||
| shape: Union[None, MultiPolygon, Polygon] = None, | ||
| shape_crs: CRS = CRS.from_user_input("EPSG:4326"), | ||
| shapefile: Union[None, str, Path] = None, |
There was a problem hiding this comment.
Indentation alignment
| self, | ||
| in_list: Sequence[ | ||
| Union[str, Raster, RasterGeom, MeshGeom, MultiPolygonGeom, PolygonGeom] | ||
| ], | ||
| base_mesh: Mesh = None, | ||
| zmin: float = None, | ||
| zmax: float = None, | ||
| nprocs: int = None, | ||
| chunk_size: int = None, | ||
| overlap: int = None, | ||
| verbosity: int = 0, | ||
| base_shape: Union[Polygon, MultiPolygon] = None, | ||
| base_shape_crs: Union[str, CRS] = "EPSG:4326", |
There was a problem hiding this comment.
argument and body indentation alignment
| self, | ||
| shape: Union[MultiPolygon, Polygon] = None, | ||
| level: Union[Tuple[float, float], float] = None, | ||
| contour_defn: Union[FilledContour, Contour] = None, | ||
| patch_defn: Patch = None, | ||
| shapefile: Union[None, str, Path] = None, |
There was a problem hiding this comment.
Just marking the same indentation thing since I noticed it!
| self, | ||
| raster: Union[Raster, str, os.PathLike], | ||
| zmin=None, | ||
| zmax=None, |
|
|
||
| def get_multipolygon( # type: ignore[override] | ||
| self, zmin: float = None, zmax: float = None) -> MultiPolygon: | ||
| self, zmin: float = None, zmax: float = None |
There was a problem hiding this comment.
indentation alignment thing again! Maybe I should ignore this?
| self, | ||
| multipolygon: Union[MultiPolygon, Polygon], | ||
| expansion_rate: float = None, | ||
| target_size: float = None, | ||
| nprocs: int = None, |
There was a problem hiding this comment.
Indentation alignment
| # if self.hmin is not None: | ||
| # values[np.where(values < self.hmin)] = self.hmin |
There was a problem hiding this comment.
extra spaces after #
| self, | ||
| feature: Union[LineString, MultiLineString], | ||
| expansion_rate: float, | ||
| target_size: float = None, | ||
| max_verts=200, | ||
| *, # kwarg-only comes after this | ||
| pool: Pool, |
There was a problem hiding this comment.
indentation alignement thing again
| # if self.hmin is not None: | ||
| # values[np.where(values < self.hmin)] = self.hmin |
| self, | ||
| window: rasterio.windows.Window = None, | ||
| marche: bool = False, | ||
| verbosity=None, |
There was a problem hiding this comment.
Marking for similar comment
| self, | ||
| func=lambda i: i / 2.0, | ||
| upper_bound=np.inf, | ||
| lower_bound=-np.inf, | ||
| value_type: str = "min", | ||
| rate=0.01, |
There was a problem hiding this comment.
Indentation alignment
| self, | ||
| multipolygon: Union[MultiPolygon, Polygon], | ||
| expansion_rate: float = None, | ||
| target_size: float = None, | ||
| nprocs: int = None, |
There was a problem hiding this comment.
Indentation alignment
| self, | ||
| level: Union[List[float], float], | ||
| expansion_rate: float, | ||
| target_size: float = None, | ||
| nprocs: int = None, |
There was a problem hiding this comment.
Indenatation alignment
| self, | ||
| in_list: Sequence[Union[str, Raster, Mesh, HfunRaster, HfunMesh]], | ||
| base_mesh: Mesh = None, | ||
| hmin: float = None, | ||
| hmax: float = None, | ||
| nprocs: int = None, | ||
| verbosity: int = 0, | ||
| method: str = "exact", | ||
| base_as_hfun: bool = True, | ||
| base_shape: Union[Polygon, MultiPolygon] = None, | ||
| base_shape_crs: Union[str, CRS] = "EPSG:4326", |
There was a problem hiding this comment.
Indentation alignement
| self, | ||
| value, | ||
| upper_bound=np.inf, | ||
| lower_bound=-np.inf, | ||
| value_type: str = "min", | ||
| rate=0.01, | ||
| source_index: Union[List[int], int, None] = None, |
There was a problem hiding this comment.
Indentation alignement
| self, | ||
| func=lambda i: i / 2.0, | ||
| upper_bound=np.inf, | ||
| lower_bound=-np.inf, | ||
| value_type: str = "min", | ||
| rate=0.01, | ||
| source_index: Union[List[int], int, None] = None, |
There was a problem hiding this comment.
Indentation alignement
| self, | ||
| level: Union[List[float], float] = None, | ||
| expansion_rate: float = 0.01, | ||
| target_size: float = None, | ||
| contour_defn: Contour = None, |
There was a problem hiding this comment.
Indentation alignment with body
| self, | ||
| level: float = 0, | ||
| width: float = 1000, # in meters | ||
| target_size: float = 200, | ||
| expansion_rate: float = None, | ||
| tolerance: Union[None, float] = 50, | ||
| channel_defn=None, |
There was a problem hiding this comment.
Indentation alignment
| # self._validate_raster_local( | ||
| # Raster(target_path), tmpraster.md5 | ||
| # os.copyfile() | ||
| # tgtraster.save(target_path) |
There was a problem hiding this comment.
Extra space after #. note not to remove code indentation
| from ocsmesh.mesh.base import BaseMesh | ||
| from ocsmesh.mesh.parsers import grd, sms2dm | ||
| from ocsmesh.raster import Raster | ||
| from pyproj import CRS, Transformer | ||
| from scipy.interpolate import RectBivariateSpline, RegularGridInterpolator | ||
| from shapely.geometry import LineString, MultiPolygon, Polygon, box | ||
| from shapely.ops import linemerge, polygonize |
There was a problem hiding this comment.
Separate third party from ocsmesh
| self, | ||
| threshold=0.0, | ||
| land_ibtype=0, | ||
| interior_ibtype=1, |
There was a problem hiding this comment.
body argument alignment
| # ocean_boundaries = utils.sort_edges(ocean_boundary) | ||
| # land_boundaries = utils.sort_edges(land_boundary) |
| self, | ||
| path: Union[str, os.PathLike], | ||
| overwrite: bool = False, | ||
| format="grd", # pylint: disable=W0622 |
There was a problem hiding this comment.
body arg alignment
| self, | ||
| dem_files: Union[None, Sequence[Union[str, os.PathLike]]], | ||
| out_file: Union[str, os.PathLike], | ||
| out_format: str = "shapefile", | ||
| mesh_file: Union[str, os.PathLike, None] = None, | ||
| mesh_multipolygon: Union[MultiPolygon, Polygon] = None, | ||
| ignore_mesh_final_boundary: bool = False, | ||
| zmin: Union[float, None] = None, | ||
| zmax: Union[float, None] = None, | ||
| chunk_size: Union[int, None] = None, | ||
| overlap: Union[int, None] = None, | ||
| nprocs: int = -1, | ||
| out_crs: Union[str, CRS] = "EPSG:4326", | ||
| base_crs: Union[str, CRS] = None, |
There was a problem hiding this comment.
argument body alignment
| # priority_args = [] | ||
| # for priority, dem_file in zip(priorities, dem_files): | ||
| # priority_args.append( | ||
| # (priority, temp_dir, dem_file, chunk_size, overlap)) | ||
| # | ||
| # with Pool(processes=nprocs) as p: | ||
| # p.starmap(self._process_priority, priority_args) | ||
| # p.join() |
There was a problem hiding this comment.
Remove extra space due to shift
| self, | ||
| path: Union[str, os.PathLike], |
| self, | ||
| priority: int, | ||
| temp_dir: Union[str, os.PathLike], | ||
| dem_path: Union[str, os.PathLike], | ||
| chunk_size: Union[int, None] = None, | ||
| overlap: Union[int, None] = None, |
| self, | ||
| base_mesh_path: Union[str, os.PathLike, None], | ||
| temp_dir: Union[str, os.PathLike], | ||
| priorities: Sequence[int], | ||
| dem_files: Sequence[Union[str, os.PathLike]], | ||
| z_info: dict = {}, | ||
| chunk_size: Union[int, None] = None, | ||
| overlap: Union[int, None] = None, |
| # priority_geodf = gpd.GeoDataFrame( | ||
| # columns=['geometry'], | ||
| # crs=self._calc_crs) | ||
| # for p in range(priority): | ||
| # higher_pri_path = ( | ||
| # pathlib.Path(temp_dir) / f'dem_priority_{p}.feather') | ||
| # | ||
| # if higher_pri_path.is_file(): | ||
| # priority_geodf = priority_geodf.append( | ||
| # self._read_to_geodf(higher_pri_path)) | ||
| # | ||
| # if len(priority_geodf): | ||
| # op_res = priority_geodf.unary_union | ||
| # pri_mult_poly = MultiPolygon() | ||
| # if isinstance(op_res, MultiPolygon): | ||
| # pri_mult_poly = op_res | ||
| # else: | ||
| # pri_mult_poly = MultiPolygon([op_res]) | ||
| # | ||
| # | ||
| # if rast_box.within(pri_mult_poly): | ||
| # _logger.info( | ||
| # f"{dem_path} is ignored due to priority...") | ||
| # continue | ||
| # | ||
| # if rast_box.intersects(pri_mult_poly): | ||
| # _logger.info( | ||
| # f"{dem_path} needs clipping by priority...") | ||
| # | ||
| # # Clipping raster can cause problem at | ||
| # # boundaries due to difference in pixel size | ||
| # # between high and low resolution rasters | ||
| # # so instead we operate on extracted polygons | ||
| # geom_mult_poly = geom_mult_poly.difference( | ||
| # pri_mult_poly) |
There was a problem hiding this comment.
extra spaces due to shift
| self, | ||
| base_mesh_path: Union[str, os.PathLike, None], | ||
| temp_dir: Union[str, os.PathLike], | ||
| priority: int, | ||
| dem_file: Union[str, os.PathLike], | ||
| z_info: dict = {}, | ||
| chunk_size: Union[int, None] = None, | ||
| overlap: Union[int, None] = None, |
| self, | ||
| dem_files: Sequence[Union[str, os.PathLike]], | ||
| out_file: Union[str, os.PathLike], | ||
| out_format: str = "shapefile", | ||
| mesh_file: Union[str, os.PathLike, None] = None, | ||
| hmin: Union[float, None] = None, | ||
| hmax: Union[float, None] = None, | ||
| contours: List[List[float]] = None, | ||
| constants: List[List[float]] = None, | ||
| chunk_size: Union[int, None] = None, | ||
| overlap: Union[int, None] = None, | ||
| method: str = "exact", | ||
| nprocs: int = -1, |
There was a problem hiding this comment.
Arg body alignment
| self, | ||
| path: Union[str, os.PathLike], | ||
| crs: Union[str, CRS] = None, | ||
| chunk_size=None, | ||
| overlap=None, |
There was a problem hiding this comment.
body arg alignment
| self, | ||
| hmin=None, | ||
| zmin=None, | ||
| zmax=None, | ||
| window=None, | ||
| overlap=None, | ||
| band=1, |
There was a problem hiding this comment.
body arg alignment
| self, | ||
| band=1, | ||
| window=None, | ||
| axes=None, | ||
| vmin=None, | ||
| vmax=None, | ||
| cmap="topobathy", | ||
| levels=None, | ||
| show=False, | ||
| title=None, | ||
| figsize=None, | ||
| colors=256, | ||
| cbar_label=None, | ||
| norm=None, | ||
| **kwargs, |
There was a problem hiding this comment.
body arg alignment
| # # Write orignal band | ||
| # dst.write_band(i + n_bands_new // 2, outband) |
There was a problem hiding this comment.
extra spaces due to shift
| self, | ||
| level: float = 0, | ||
| width: float = 1000, # in meters | ||
| tolerance: Union[None, float] = None, |
There was a problem hiding this comment.
body arg alignment
| mesh: jigsaw_msh_t, | ||
| shape: Union[box, Polygon, MultiPolygon], | ||
| from_box: bool = False, |
There was a problem hiding this comment.
body arg alignment
| mesh: jigsaw_msh_t, | ||
| shape: Union[box, Polygon, MultiPolygon], |
There was a problem hiding this comment.
body arg alignment
| mesh: jigsaw_msh_t, | ||
| shape: Union[box, Polygon, MultiPolygon], | ||
| use_box_only: bool = False, | ||
| fit_inside: bool = True, | ||
| inverse: bool = False, | ||
| in_place: bool = False, | ||
| check_cross_edges: bool = False, |
There was a problem hiding this comment.
argument body alignment
| mesh: jigsaw_msh_t, | ||
| vert_in: Sequence[int], | ||
| can_use_other_verts: bool = False, | ||
| inverse: bool = False, | ||
| in_place: bool = False, |
There was a problem hiding this comment.
argument body alignment
| *mesh_list, | ||
| out_crs="EPSG:4326", | ||
| drop_by_bbox=True, | ||
| can_overlap=True, | ||
| check_cross_edges=False, |
There was a problem hiding this comment.
argument body alignment
SorooshMani-NOAA
left a comment
There was a problem hiding this comment.
Hi @yosoyjay, thank you for the changes. It looks much cleaner. I just have a couple of concerns that is vividly visible from my comments on the code:
- Some function/method arguments are indented such that the indentation aligns with the body
- Some commented lines are shifted and the commented code are not indented properly and there's extra space
- Some calls spread the arguments (1 per line) and sometimes it's too much.
- Some lines seem to have become very long.
Otherwise I really liked these changes. It would be useful to add it as Github actions so that it's also clear how to run checks locally before submitting the code to repo.
I also have another concern. I made a lot of changes for adding docstring in enhance/docstring branch. I think there would be a lot of conflicts for merging, so I was wondering how much of your changes are manual and how much automatic. In case most things are automatic, we can first merge docstring branch to main and then apply changes to that branch. I'll address your questions in a separate comment.
|
For the question of those specific errors (line numbers are based on files in the pull request branch):
|
This pull request is to improve the linting for the package in response to issue #105 in ODSSM.
The current PR includes changes resulting the application of
isortandblackand small change insetup.pyto improve the robustness of the version check for the C++ complier (-dumpversionvs.--version) .flake8compliance is a work in progress.flake8is currently configured insetup.cfgto ignore E501 (line length), E226 (lack of white space around binary operators), and E503 (line break before binary operator, but other errors remain that I'd like feedback on:Once the
flake8issue is resolved, I'd be in favor of integrating tests forisort,black, andflake8as part of the Github Action workflow for testing.