Talk:Physical measurement - Mesure physique - Revision history

Furien: Protected "Talk:Physical measurement - Mesure physique": Vote on physical mesurement ([Edit=Allow only administrators] (expires 05:36, 1 July 2024 (UTC)) [Move=Allow only administrators] (expires 05:36, 1 July 2024 (UTC)))

2024-06-24T05:36:25Z

Protected "Talk:Physical measurement - Mesure physique": Vote on physical mesurement ([Edit=Allow only administrators] (expires 05:36, 1 July 2024 (UTC)) [Move=Allow only administrators] (expires 05:36, 1 July 2024 (UTC)))

EricMadelaine at 20:09, 23 June 2024

2024-06-23T20:09:07Z

Furien at 14:44, 22 June 2024

2024-06-22T14:44:56Z

Furien at 08:04, 15 June 2024

2024-06-15T08:04:30Z

Furien at 08:01, 15 June 2024

2024-06-15T08:01:33Z

Furien at 08:00, 15 June 2024

2024-06-15T08:00:22Z

EricMadelaine: I propose to separate 2 kinds of physical measurements, single-time measurements from time-series, to make them simpler and more precise.

2024-06-09T17:21:11Z

I propose to separate 2 kinds of physical measurements, single-time measurements from time-series, to make them simpler and more precise.

New page

As promissed to Frederic, I will comment his "Physical measurement" proposal here, and present an alternative solution.

1) history:
This entity was introduced shortly in 2022, but not part of the voted items of the KarstLink core.
We elaborated together, then presented at the UIS congress (paper cosigned by Frederic Urien, Peter Matthews and myself, Eric Madelaine) a proposal for an extension dedicated to measurements by (mainly underground) sensors as time-series, that many cavers use for temperature, pressure, conductivity, gaz rates, etc.
Here monitoring can mean short term to very long series of measurements for a single campaign, typically weeks, or several months, as opposed to individual measures as one could take typically with a and-held thermometer.
In April/May 2024, related to the planned implementation of a physical measurement database module in the Grottocenter plateform, Frederic introduced in the karstlink wiki his proposal for Physical Measurements, that covers in a single Entity the two kinds of measures.

2) Encoding of physical measurements versus time-series:
In addition to the obvious differences in the devices and in the way the measures are taken, the structure of
the results of observations is quite different. This shows clearly in the item of proposal 1 about results, defined as "made up of one or more Time/Value pairs". My point is (please refer to the UIS congress communication for more details) that when you have to deal with a (time-series) containing typically a hundred thousand sets of quantities (e.g. temperature, pressure, conductivity) at each time when the sensor registered a measure, you definitely do not want to build this complex object as a huge data structure that you would store in your database. In practice, the device providers encode this data in compact formats (usually text formats), and what is important to store, in addition to this text file, is its structure (format of timestamps, and which quantity/unit is in each column)

3) I find Frederic proposal quite complicated because he wants to define a single ontology entity for both a single measurement and a time-series, that forces him to underspecify the relations within the entity. One typical case is with time stamps, where a single measurement would be _necessarily_ associated to a DateTimeStamp, and nothing else, while a time series would certainly be _necessarily_ associated to a start date and an end date, that would allow applications using such data to make requests based on these start/end dates, without having to scan the full list of time/value pairs...

4) My proposal (B) is then to separate these 2 kinds of physical measurements. Of course there is a subset that will be common to the two "ObservationTypes", but each of them will be simpler and more precise.

5) In addition, from my experience using such sensors, it should be very useful to add a notion of "quality" of the measures, that are usualy coming from the caracteristics of the sensors, but also from poor calibration procedures, for exemple. When people will extract sensor data from our database, say 20 years from now, to make analyses of the effect of climate changes, this will be an important information. I have no precise idea what this "quality" could be, if someone has references to existing ontologies, I will be interested.

@@ Line 18: / Line 18: @@
 **Response from Frédéric
 ) the physical measurements were presented succinctly in 2020. In 2022 Eric punctually updated the initial presentation
@@ Line 24: / Line 26: @@
 ) The data structure proposed for the standardized format of Proposition 1 is composed of a series of timestaps and values, as for the data provided by the sensor. The difference is that each sensor provides data in a different format which will require each user to transform the data.
-Providing standardized data or offering a file produced by the sensor presents a very important difference: the standardized file will only provide the requested data whereas the sensor file contains all the data collected, including data that does not correspond to the request for the user
+Providing standardized data or offering a file produced by the sensor presents a very important difference: the standardized file will only provide the requested data whereas the sensor file contains all the data collected, including data that does not correspond to the request for the user.
 ) Eric's proposal is incomplete to allow the exchange of data, which is the objective of Karstlink: information is missing on the license, on the author, on the organization which made the data available.
 It seems to me that Eric's proposal includes elements that were defined in proposal 1, by not retaining these mandatory elements and by not allowing data in a standardized format to be made available
-) There is no difference between a single value and a series of a million measurements at the metadata level. The only difference is that the single measurement will not be associated with a file corresponding to the original data. Proposition 1 would not be different if it allowed describing a single measurement or measurements coming from a sensor, except for the point indicated. It is not useful to construct 2 different entities
+==> this point was refering to a preliminary very incomplete version of Proposal B, it has been added, namely in the *common* part of the physical measurement subclass.
 ) The notion of quality cannot be defined with a single criterion: part of it is linked to the sensor with a number of more or less important criteria which can intervene in the quality of the data. There is also a human factor which intervenes and which will lead to imprecise data.
 In proposition 1 it's possible for all people to put a comment / a description (by managing copyright). This will allow the author, then other users, to give an opinion on the quality of the data but also to make other information available in a single field. Putting a specific text field for quality does not seem to me to be a satisfactory solution.

@@ Line 17: / Line 17: @@
 ) In addition, from my experience using such sensors, it should be very useful to add a notion of "quality" of the measures, that are usualy coming from the caracteristics of the sensors, but also from poor calibration procedures, for exemple. When people will extract sensor data from our database, say 20 years from now, to make analyses of the effect of climate changes, this will be an important information. I have no precise idea what this "quality" could be, if someone has references to existing ontologies, I will be interested.
-**Response from Frédéric **
+**Response from Frédéric
 ) the physical measurements were presented succinctly in 2020. In 2022 Eric punctually updated the initial presentation

@@ Line 21: / Line 21: @@
 ) the physical measurements were presented succinctly in 2020. In 2022 Eric punctually updated the initial presentation
-Physical_mesurement.png|Copying the history of the physical measurements page
+[[File:Physical_mesurement.png|Copying the history of the physical measurements page]]
 ) The data structure proposed for the standardized format of Proposition 1 is composed of a series of timestaps and values, as for the data provided by the sensor. The difference is that each sensor provides data in a different format which will require each user to transform the data.

@@ Line 20: / Line 20: @@
 ) the physical measurements were presented succinctly in 2020. In 2022 Eric punctually updated the initial presentation
-<gallery>
 Physical_mesurement.png|Copying the history of the physical measurements page
 ) The data structure proposed for the standardized format of Proposition 1 is composed of a series of timestaps and values, as for the data provided by the sensor. The difference is that each sensor provides data in a different format which will require each user to transform the data.

← Older revision		Revision as of 08:00, 15 June 2024
Line 16:		Line 16:

	5) In addition, from my experience using such sensors, it should be very useful to add a notion of "quality" of the measures, that are usualy coming from the caracteristics of the sensors, but also from poor calibration procedures, for exemple. When people will extract sensor data from our database, say 20 years from now, to make analyses of the effect of climate changes, this will be an important information. I have no precise idea what this "quality" could be, if someone has references to existing ontologies, I will be interested.		5) In addition, from my experience using such sensors, it should be very useful to add a notion of "quality" of the measures, that are usualy coming from the caracteristics of the sensors, but also from poor calibration procedures, for exemple. When people will extract sensor data from our database, say 20 years from now, to make analyses of the effect of climate changes, this will be an important information. I have no precise idea what this "quality" could be, if someone has references to existing ontologies, I will be interested.
		+
		+	Response from Frédéric
		+
		+	1) the physical measurements were presented succinctly in 2020. In 2022 Eric punctually updated the initial presentation
		+	<gallery>
		+	Physical_mesurement.png\|Copying the history of the physical measurements page
		+	</gallery>
		+
		+	2) The data structure proposed for the standardized format of Proposition 1 is composed of a series of timestaps and values, as for the data provided by the sensor. The difference is that each sensor provides data in a different format which will require each user to transform the data.
		+	Providing standardized data or offering a file produced by the sensor presents a very important difference: the standardized file will only provide the requested data whereas the sensor file contains all the data collected, including data that does not correspond to the request for the user
		+
		+	3) Eric's proposal is incomplete to allow the exchange of data, which is the objective of Karstlink: information is missing on the license, on the author, on the organization which made the data available.
		+	It seems to me that Eric's proposal includes elements that were defined in proposal 1, by not retaining these mandatory elements and by not allowing data in a standardized format to be made available
		+
		+	4) There is no difference between a single value and a series of a million measurements at the metadata level. The only difference is that the single measurement will not be associated with a file corresponding to the original data. Proposition 1 would not be different if it allowed describing a single measurement or measurements coming from a sensor, except for the point indicated. It is not useful to construct 2 different entities
		+
		+	5) The notion of quality cannot be defined with a single criterion: part of it is linked to the sensor with a number of more or less important criteria which can intervene in the quality of the data. There is also a human factor which intervenes and which will lead to imprecise data.
		+	In proposition 1 it's possible for all people to put a comment / a description (by managing copyright). This will allow the author, then other users, to give an opinion on the quality of the data but also to make other information available in a single field. Putting a specific text field for quality does not seem to me to be a satisfactory solution.

← Older revision	Revision as of 05:36, 24 June 2024
(No difference)